VictoriaMetrics Operator

Update note 1: vmagent, vmalert, vmalertmanager, vmsingle: the config-reloader sidecar now gets an extra --target-dir= argument per watched directory, which changes the pod spec. All VMAlertmanager pods, and every VMAgent, VMAlert or VMSingle pod whose reloader watches a directory (spec.configMaps, relabeling or stream aggregation configs, rule ConfigMaps), will be rolled out once during this upgrade.

Update note 2: vmcluster: a serviceSpec with useAsDefault: true that makes the default Service of vmstorage non-headless, that is a spec.type other than ClusterIP, or an explicit spec.clusterIP/spec.clusterIPs other than None, is now rejected at spec.vmstorage.serviceSpec. Reconciliation of such a VMCluster fails until the serviceSpec keeps the Service headless, or until useAsDefault is removed, which turns it into an additional Service. See #2491.

Update note 3: vmcluster, vmalertmanager: spec.vmselect.serviceSpec of VMCluster and spec.serviceSpec of VMAlertmanager with useAsDefault: true and an explicit spec.type, but no spec.clusterIP, now produce a Service with a virtual IP instead of a headless one. That Service is recreated during the upgrade and gets a new clusterIP. For a VMCluster with spec.requestsLoadBalancer.enabled, the recreated Service is the vmauth proxy of vmselect, while its internal Service stays headless. Add spec.clusterIP: None, or remove spec.type, to keep the previous behaviour. See #2487.

Update note 4: vmrule, vmuser, vmalertmanagerconfig, vmanomalyconfig, vmservicescrape, vmpodscrape, vmnodescrape, vmprobe, vmstaticscrape, vmscrapeconfig: status.updateStatus of these config-only objects no longer reports whether the applications selecting them accepted their content. It is now operational unless the operator cannot parse the object's own spec, in which case it is set to failed. The ignored value is no longer set. The state reported by each application is available at status.conditions.

• Dependency: vmoperator: Updated default versions for VM apps to v1.152.0.

• Dependency: vmoperator: Updated default versions for VT apps to v0.11.1.

• Dependency: vmoperator: Updated default version for VMAnomaly to v1.30.7.

• Dependency: vmoperator: Updated default version for Alertmanager to v0.34.1.

• FEATURE: vmanomaly: support bounded spec.reader.workers, query-level business policies, and settings.native_threads_per_worker introduced in vmanomaly v1.30.2. See #2496.

• FEATURE: vmanomaly: add typed spec.writer.batchMaxSeries, spec.writer.batchMaxBytes, and spec.writer.metricPrefixCacheMaxEntries fields for the bounded writer controls introduced in vmanomaly v1.30.3. See #2542.

• FEATURE: vmoperator: add k8s_version label with the Kubernetes server version to the vm_app_version metric. See #2530.

• FEATURE: vmoperator: add operator_controller_errors_total{controller,namespace,name,reason} metric, which counts reconcile errors of every controller by reason: get_object, parse_object, cancel_context, conflict, declined and other. It deprecates operator_controller_object_get_errors_total, operator_controller_object_parsing_errors_total, operator_controller_reconcile_conflict_errors_total and operator_controller_reconcile_errors_total, which are still exported, but will be removed in a future release. See #2536.

• FEATURE: vtagent: add VTAgent CRD for vtagent. It can accept OTLP trace spans over HTTP and gRPC and replicates them to one or more VictoriaTraces instances. The new VTAgent CRD must be installed in order to use it. See #2510.

• FEATURE: vtagent, vtsingle, vtcluster: add grpcSpec field (spec.insert.grpcSpec for VTCluster) to accept OTLP trace spans over gRPC in addition to HTTP. Its listenPort is exposed at the component Service and can be secured with the optional tlsConfig. See #2510.

• FEATURE: vlagent, vlsingle, vlcluster: add minVersion and cipherSuites fields to tlsConfig of syslog TCP listeners. See #2510.

• FEATURE: vlsingle, vtsingle: add removePvcAfterDelete field, which makes Kubernetes garbage-collect the PersistentVolumeClaim together with the deleted VLSingle or VTSingle object. It is disabled by default, so the PersistentVolumeClaim with the stored data is now preserved after the object deletion. Previously, it was always removed together with the object. See #2545.

• FEATURE: vmcluster, vmalertmanager: allow a serviceSpec with useAsDefault: true to replace the headless default Service of vmselect and VMAlertmanager with a regular one by defining spec.type. Previously, spec.type: ClusterIP still inherited clusterIP: None from the default Service, so the only way to get a virtual IP was to pin spec.clusterIP by hand. See #2487.

• BUGFIX: vmoperator: set status.updateStatus for config-only objects - VMRule, VMUser, VMAlertmanagerConfig, VMAnomalyConfig, VMServiceScrape, VMPodScrape, VMNodeScrape, VMProbe, VMStaticScrape and VMScrapeConfig - even when no application selects them. Previously, such objects were left without any status, which failed health checks of GitOps tools such as ArgoCD. The status is now operational, or failed if the operator cannot parse the object's spec. See #2649.

• BUGFIX: vmagent, vmanomaly: set the default value 0 to spec.shardCount in the CRD schema. Previously, the field was absent unless sharding was configured, so every read of the object's /scale subresource failed with the spec replicas field ".spec.shardCount" does not exist, which broke spec.vpa, spec.hpa and any external autoscaler pointed at the object. The VMAgent and VMAnomaly CRDs must be updated to get this fix. See #2473.

• BUGFIX: vmoperator: set default values for each possible level label of operator_log_messages_total metric. See #2477.

• BUGFIX: vmoperator: add the default preStop sleep hook only on Kubernetes >= 1.30. Previously it was added on 1.29 as well, where the PodLifecycleSleepAction feature gate is alpha and off by default, the affected compon

…(truncated)

• BUGFIX: vldistributed: add validation webhook for VLDistributed resources.

Update note 1: vmoperator: validation webhooks for prometheus-operator CRDs added to the operator in 0.72.0 is now disabled by default. Use VM_PROMCR_VALIDATION_ENABLED env var to enable it explicitly.

Update note 2: vmoperator: the operator's ClusterRole must be updated to grant get, list, watch, create, update and delete on networking.k8s.io/networkpolicies. The operator now looks up NetworkPolicy objects on every reconcile, even when networkPolicy is not set. With an older ClusterRole, reconciliation of all VMSingle, VMAgent, VMAlert, VMAlertmanager, VMAuth, VMAnomaly, VMCluster, VLSingle, VLAgent, VLCluster, VTSingle and VTCluster objects fails with a Forbidden error. See helm-charts#3129.

• Dependency: vmoperator: Updated default versions for VM apps to v1.148.0 version

• Dependency: vmoperator: Updated default versions for VL apps to v1.52.0.

• Dependency: vmoperator: Updated default versions for VT apps to v0.10.0 version.

• Dependency: vmoperator: Updated default versions for VMAnomaly to v1.30.0 version

• FEATURE: vmanomaly: support Temporal Envelope models, online-model history strength and warmup settings, causal and complexity-aware autotune options, and separate reader fetch and processing controls introduced in vmanomaly v1.30.0. See #2440.

• FEATURE: vmsingle, vmagent, vmalert, vmalertmanager, vmauth, vmcluster, vmanomaly, vlsingle, vlagent, vlcluster, vtsingle, vtcluster: add networkPolicy field, including every cluster component and requestsLoadBalancer. When set, the operator creates a NetworkPolicy for the component's pods with the given ingress and egress rules, and deletes it when the field is removed. Only directions with rules are restricted, and an empty networkPolicy: {} blocks all ingress traffic. See helm-charts#2977 and #2299.

• FEATURE: vmauth: add opt-in spec.waitForConfigReload field. When enabled, the operator confirms that every replica has actually picked up a config change before marking dependent VMUser objects as applied, so their status can be trusted to reflect what's actually running rather than what was merely written out. vmdistributed's internal VMAuth load balancer always waits for this confirmation before rotating a zone in or out of the backend list, regardless of spec.vmAuth.spec.waitForConfigReload, so traffic is never routed to a zone that's still running a stale config.

• FEATURE: vlagent: add basicAuth field support to remoteWrite entries. See #2371.

• FEATURE: vmcluster: support VMCluster extraStorageNodes for vmselect component.

• FEATURE: vmuser: add defaultVMAccessClaim field to spec.jwt, mapped to vmauth's jwt.default_vm_access_claim. It lets a VMUser accept JWTs that don't carry a vm_access claim, matching vmauth v1.147.0+ behavior. See #2375.

• FEATURE: helm-converter: parse the victoria-metrics-auth chart's config value (vmauth's own native config file). Each config.users entry is converted into a standalone VMUser CR, and config.unauthorized_user is converted into the VMAuth CR's spec.unauthorizedUserAccessSpec. The generated VMUser CRs are appended to the same output file as additional YAML documents, and the VMAuth CR's spec.userSelector is set to a dedicated label matching them, so the operator actually loads them (a bare VMAuth CR's default selectors match nothing). See #2397.

• FEATURE: vmoperator: introduce VLDistributed CR, which controls multiple region-distributed VictoriaLogs clusters.

• BUGFIX: vmoperator: support Ignored status for child objects that were not picked, also do not set Failed status if object was applied on at least one parent object. See #2432.

• BUGFIX: vmoperator: removed library/ component, while building CR images that do not contain / in repo name. See #2409.

• BUGFIX: vmoperator: log only the changed key names and value sizes, instead of the full values, when updating ConfigMap. Previously a single data_diff log line could reach multiple megabytes for large ConfigMaps and break downstream log pipelines with per-line size limits. See #2426.

• BUGFIX: vmagent, vmsingle: add missing list verb to config-reloader's secrets RBAC rule. See #2384.

• BUGFIX: vmoperator: return an error instead of panicking when a Labels map value is malformed (missing the = separator) during config parsing.

• BUGFIX: vmalert: when no notifiers are configured, ignore alerting rules from selected VMRules instead of failing reconciliation; recording rules in the same group are kept and still reconciled. See #2388.

• BUGFIX: vmalertmanager: default pod securityContext.fsGroup to 65534 when persistent storage is configured and neither useStrictSecurity nor a user securityContext is set. Without it the alertmanager process cannot write notification log and silences to a freshly provisioned volume, so silences are silently lost on pod restart. User-provided securityContext and useStrictSecurity keep their previous behaviour.

• BUGFIX: vmoperator: allow pinning images by digest via the image.tag field of all operator CRs. When tag is a digest (e.g. sha256:<hex>) it is now joined to the repository with @ instead of :, producing a valid repository@sha256:<digest> reference. Regular tags are unaffected.

• BUGFIX: helm-converter: fix persistentVolume.storageClassName being dropped during conversion due to a stale storageClass field name. See #2389.

• BUGFIX: helm-converter: fix securityContext.runAsNonRoot, runAsUser, runAsGroup, seccompProfile, appArmorProfile, seLinuxOptions, and windowsOptions being silently dropped during conversion; these are now promoted to spec.securityContext.podSecurityContext, matching how the operator applies them to containers. See #2391.

• BUGFIX: helm-converter: fix TLS/mTLS settings (tlsCAFile, tlsCertFile, tlsKeyFile, tlsServerName, tlsInsecureSkipVerify) on remoteWrite entries being dropped during conversion for vmagent, vlagent, vlcollector, and vmalert charts. See #2390.

• BUGFIX: helm-converter: fix cannot unmarshal object into Go struct field ... headers of type []string error when a chart's default values.yaml ships a headers: {} map (e.g. vmalert's datasource.headers, notifier.headers, remoteWrite.headers); such maps are now normalized to the operator's key:value string-slice format before conversion. See

…(truncated)

Dependency: vmoperator: Updated default versions for VM apps to v1.147.0 version

Dependency: vmoperator: Updated default versions for VT apps to v0.9.4 version.

Dependency: vmoperator: Updated default versions for VMAnomaly to v1.28.7 version

SECURITY: vmagent, vmsingle: remove cluster-wide secrets and configmaps permissions from the operator-managed ClusterRole. Secret access for the config-reloader is now granted via a namespace-scoped Role limited to the single operator-managed config secret. For vmsingle in ingest-only mode (the default), no secret or configmap permissions are granted at all.

BUGFIX: vmanomaly: add missing scatter_infer_jobs field to the periodic scheduler config struct. See https://github.com/VictoriaMetrics/operator/issues/2328

BUGFIX: vmanomaly: preserve insertion order of keys in ProphetModel seasonalities, tz_seasonalities, compression, and args fields; previously the operator re-emitted them with keys sorted alphabetically, which broke round-trips for configs that specified keys in a non-alphabetical order. Also renamed the singular seasonality/tz_seasonality YAML keys (deprecated) to the plural seasonalities/tz_seasonalities to match the vmanomaly configuration format. See https://github.com/VictoriaMetrics/operator/issues/2356

BUGFIX: vmagent, vmsingle: create a Role and RoleBinding in each namespace listed in WATCH_NAMESPACES so that vmagent/vmsingle can perform service discovery in all watched namespaces, not only its own. Previously, in namespaced mode, vmagent/vmsingle could only scrape targets from its own namespace due to missing RBAC in other watched namespaces.

BUGFIX: vmagent: fix missing credential secret and config-reloader setup in ingestOnlyMode when remote write entries carry authentication secrets (basicAuth.password, bearerTokenSecret, or oauth2.clientSecret). Previously the operator-managed secret containing the credential files was never created in ingest-only mode, causing vmagent to start with dangling file references. The secret is now reconciled and the config-reloader is configured to watch it for credential rotation.

• BUGFIX: vmrule: removed strategic-merge list semantics from rules as it breaks many rules, where it's a common practice to use a same rule name with different thresholds and severities.

• Dependency: vmoperator: Updated default versions for VM apps to v1.147.0 version

• Dependency: vmoperator: Updated default versions for VL apps to v1.51.0.

• Dependency: vmoperator: Updated default versions for VT apps to v0.9.4 version.

• SECURITY: vmagent, vmsingle: remove cluster-wide secrets and configmaps permissions from the operator-managed ClusterRole. Secret access for the config-reloader is now granted via a namespace-scoped Role limited to the single operator-managed config secret. For vmsingle in ingest-only mode (the default), no secret or configmap permissions are granted at all.

• FEATURE: vmuser, vmauth: add access_log field support. See docs and #2359.

• FEATURE: vmuser: add useExtraService boolean field to crd.namespacedName and crd.objects entries in VMUser target references. When set to true, the operator resolves the target URL using the CR's additional service (configured via spec.serviceSpec) instead of the default service. This lets you route VMAuth traffic to a dedicated service with different port mappings or service type without changing the primary service. See related types: CRDRef, NamespacedName. See #2333.

• FEATURE: vmoperator: add useLegacyNaming field to all operator CRs. When enabled, managed resources are named using the standalone Helm chart convention (CR name directly for single-component CRs; <name>-<component> for cluster CRs) instead of the default operator prefix convention. Useful when migrating from standalone charts to operator-managed resources without recreating existing resources.

• FEATURE: vmdistributed: support using VMSingle backends alongside VMCluster backends in separate zones.

• FEATURE: vmoperator: add VM_OPENSHIFT_COMPATIBILITY env var (OpenshiftCompatibilityMode config field, default auto) to enable OpenShift-specific compatibility features for managed pods. In auto mode the operator detects OpenShift at startup via presence of security.openshift.io/v1 API group. Currently covers automatic mounting of the openshift-service-ca.crt ConfigMap into VMAgent and VMSingle pods at /etc/ssl/certs/openshift-service-ca/service-ca.crt, so that targets signed with the OpenShift service signing CA could be verified without additional TLS config. Supported values: auto, enabled, disabled. See #2971.

• BUGFIX: vmcluster: fix CR getting permanently stuck in expanding status after an image update when HPA is enabled on vminsert, vmselect, or vmstorage. The operator was comparing a stale replica count (captured at reconcile start) against live status fields, so the readiness check could never pass once HPA changed the replica count mid-rollout. The same issue affected VMAuth, VLCluster, and VTCluster components with HPA enabled. See #2324.

• BUGFIX: config-reloader: fix possible panic on Secret watch events when the informer's local cache fell out of sync and Kubernetes delivered a stale tombstone entry instead of the Secret object. The config-reloader now unwraps tombstones correctly and logs an error for any other unexpected types.

• BUGFIX: vmanomaly: add missing scatter_infer_jobs field to the periodic scheduler config struct. See #2328.

• BUGFIX: vmoperator: switch default app probes to tcpSocket startupProbe when TLS is enabled on the managed HTTP endpoint. This avoids broken kubelet httpGet checks against TLS and mTLS-protected workloads. See #1824.

• BUGFIX: vmagent, vmsingle: create a Role and RoleBinding in each namespace listed in WATCH_NAMESPACES so that vmagent/vmsingle can perform service discovery in all watched namespaces, not only its own. Previously, in namespaced mode, vmagent/vmsingle could only scrape targets from its own namespace due to missing RBAC in other watched namespaces.

• BUGFIX: vmagent: fix missing credential secret and config-reloader setup in ingestOnlyMode when remote write entries carry authentication secrets (basicAuth.password, bearerTokenSecret, or oauth2.clientSecret). Previously the operator-managed secret containing the credential files was never created in ingest-only mode, causing vmagent to start with dangling file references. The secret is now reconciled and the config-reloader is configured to watch it for credential rotation.

• Dependency: vmoperator: Updated default versions for VM apps to v1.146.0 version

• Dependency: vmoperator: Updated default versions for VL apps to v1.51.0.

• Dependency: vmoperator: Updated default versions for VT apps to v0.9.3 version.

• FEATURE: vmoperator: add victoriametrics_app=true label to all metrics scraped by the operator. See #2261.

• BUGFIX: config-reloader: fix possible panic on Secret watch events when the informer's local cache fell out of sync and Kubernetes delivered a stale tombstone entry instead of the Secret object. The config-reloader now unwraps tombstones correctly and logs an error for any other unexpected types.

• BUGFIX: vmoperator: update status currentRevision and currentReplicas for StatefulSet with OnDelete update strategy. See #1242.

• BUGFIX: vmoperator: retry reconcile errors, that may lead to expanding state, before resource could hang in expanding state.

• BUGFIX: vmcluster, vlcluster and vtcluster: when storage HPA was enabled, generated -storageNode flags could become incorrect after scaling, which could break expected routing to storage nodes; now the operator derives storage node count from the current StatefulSet state so generated flags stay correct during HPA-driven scaling. See #2117.

• BUGFIX: vmoperator: update status currentRevision and currentReplicas for StatefulSet with OnDelete update strategy. See #1242.

• BUGFIX: config-reloader: fix configreloader_last_reload_success_timestamp_seconds metric to report time in seconds instead of milliseconds.

• BUGFIX: vmoperator: ignore NotFound errors, that may occur during attempt to update status on a missing resource.

• BUGFIX: vmanomaly: pass the configured TLS CA bundle to the reader, writer and monitoring clients. Previously the CA was mounted as a volume but dropped during config generation, so a tlsConfig with only a CA produced no verify_tls reference to it; insecureSkipVerify is now also propagated correctly.

• BUGFIX: config-reloader: fix missed reload for watched files whose names contain .. (e.g. rules..yaml). Previously any path containing .. was silently skipped; now only Kubernetes synthetic entries whose basename starts with .. (e.g. ..data) are ignored. See #2253.

Update note 1: vmalert: rule ConfigMaps now store gzip-compressed content in binaryData and an init container decompresses them before VMAlert starts. Existing VMAlert pods will be rolled out once during this upgrade.

• FEATURE: vmoperator: add validating webhooks for Prometheus Operator CRDs (ServiceMonitor, PodMonitor, PrometheusRule, Probe, ScrapeConfig, AlertmanagerConfig). Each object is converted to its VM equivalent and validated when webhooks are enabled. See #2270.

• FEATURE: vmscrapeconfig: add support for consulAgentSDConfigs, dockerSDConfigs, dockerSwarmSDConfigs, marathonSDConfigs, and yandexCloudSDConfigs service discovery types, bringing VMScrapeConfig to full parity with VictoriaMetrics sd_configs. See #2265.

• FEATURE: vmalert: rule ConfigMaps now store gzip-compressed rule files in binaryData, allowing larger rule sets within Kubernetes object size limits. An init container decompresses the rules before VMAlert starts.

• FEATURE: vmscrapeconfig: add support for consulAgentSDConfigs, dockerSDConfigs, dockerSwarmSDConfigs, marathonSDConfigs, and yandexCloudSDConfigs service discovery types, bringing VMScrapeConfig to full parity with VictoriaMetrics sd_configs. See #2265.

• FEATURE: vmoperator: add victoriametrics_app=true label to all metrics scraped by the operator. See #2261.

• BUGFIX: vmoperator: skip reconciliation only for CRs with genuine spec parse errors; CRs whose ParsingSpecError is caused solely by unknown fields (e.g. after an operator downgrade) are now reconciled normally instead of being silently skipped.

• BUGFIX: vmoperator: fix potential deadlock in operator_object_status metrics collector when the number of tracked objects exceeds 250. The Collect method previously held a mutex while sending to the prometheus channel, which could deadlock if the channel was full and another goroutine was waiting on the same mutex. See #2239.

• BUGFIX: config-reloader: fix missed reload for watched files whose names contain .. (e.g. rules..yaml). Previously any path containing .. was silently skipped; now only Kubernetes synthetic entries whose basename starts with .. (e.g. ..data) are ignored. See #2253.

• BUGFIX: vmoperator: fix potential deadlock in operator_object_status metrics collector when the number of tracked objects exceeds 250. The Collect method previously held a mutex while sending to the prometheus channel, which could deadlock if the channel was full and another goroutine was waiting on the same mutex. See #2239.

Update note 1: the new default preStop hook causes a rolling update of all applicable pods on operator upgrade (on Kubernetes >= 1.29). To avoid this, set VM_ENABLE_DEFAULT_PRESTOP_HOOK=false on the operator before upgrading. Once the upgrade is complete, you can re-enable it by removing the override (or setting it to true) to roll out the hook at a time of your choosing. Alternatively, disable the hook per resource by setting spec.preStopSleepSeconds: 0.

• Dependency: vmoperator: Updated default versions for VM apps to v1.145.0 version

• FEATURE: vmoperator: added VM_COMMON_LABELS and VM_COMMON_ANNOTATIONS environment variables to apply common labels/annotations to all Kubernetes resources managed by the operator. These cannot override labels/annotations already set by the operator or via spec.managedMetadata. This also ensures HTTPRoutes and PVCs include ManagedMetadata labels and annotations

• FEATURE: vmoperator: support enableServiceLinks property in all CRs. See #2194.

• FEATURE: vmalertmanagerconfig: add url_file and alert_source_token_file fields to IncidentioConfig, as file-based alternatives to url and alert_source_token. See #2222.

• FEATURE: vmoperator: add status metrics for objects managed by each controller. See #2238.

• FEATURE: vmsingle: added spec.downsampling for structured downsampling configuration and spec.retentionFilters for structured retention filters configuration. Both require an enterprise license.

• FEATURE: vmcluster: added spec.downsampling for structured downsampling configuration (applied to both vmselect and vmstorage) and spec.vmstorage.retentionFilters for structured retention filters configuration. Both require an enterprise license.

• FEATURE: vmoperator: add default preStop lifecycle hook (15s sleep) to applicable component pods to prevent traffic loss during pod termination. Requires Kubernetes >= 1.29. Storage components (VMStorage, VLStorage, VTStorage) are excluded since their clients handle retries. The sleep duration is configurable via spec.preStopSleepSeconds on any CR; set to 0 to disable. See #1834.

• FEATURE: vmanomaly: add missing configuration parameters for vmanomaly config parity: spec.reader.offset and per-query offset, spec.writer.connectionRetryAttempts, spec.server.useReaderConnectionSettings, settings.logger_levels, and the exact/infer_every parameters of the backtesting scheduler. Previously these keys were rejected by strict config parsing.

• FEATURE: vmcluster: add spec.discovery field with enabled, interval and filter properties to configure automatic vmstorage node discovery for vminsert and vmselect. The field can be set globally or overridden per component via spec.vminsert.discovery and spec.vmselect.discovery. This is an enterprise feature and requires a valid license key. See automatic vmstorage discovery.

• BUGFIX: vmoperator: update status currentRevision and currentReplicas for StatefulSet with OnDelete update strategy. See #1242.

• BUGFIX: config-reloader: fix configreloader_last_reload_success_timestamp_seconds metric to report time in seconds instead of milliseconds.

• BUGFIX: vmoperator: enable strict CR spec unmarshalling when creating objects. See #2882.

• BUGFIX: vmoperator: ignore NotFound errors, that may occur during attempt to update status on a missing resource.

• BUGFIX: vmagent, vmanomaly: fix VPA scale subresource lookup failure when spec.shardCount is unset by always reporting at least 1 in status.shards. See #2229.

• BUGFIX: vmagent: fix HPA targeting the underlying Deployment/StatefulSet (pod replicas) instead of the VMAgent CR scale subresource (spec.shardCount); HPA now correctly scales the number of shards. See #2229.

• BUGFIX: vmanomaly: emit the OnlineQuantileModel smoothing parameter under its correct key global_smoothing instead of the unrecognized global_smooth, which vmanomaly silently ignored.

• BUGFIX: vmanomaly: pass the configured TLS CA bundle to the reader, writer and monitoring clients. Previously the CA was mounted as a volume but dropped during config generation, so a tlsConfig with only a CA produced no verify_tls reference to it; insecureSkipVerify is now also propagated correctly.

• BUGFIX: vmanomaly: serialize the tz timezone of spec.reader, of each query, and of the periodic scheduler as a string. It was serialized to {} and could not be parsed from a string, so any configuration that set tz failed to reconcile.

• BUGFIX: vmanomaly: fix the AutoTunedModel trials parameter spelling n_trials, previously it was rendered as n_trails which was silently ignored by vmanomaly.

• BUGFIX: vmanomaly: preserve an explicit 0.0 for the anomaly_score_outside_data_range parameter (both settings and model level), which was dropped by the underlying float field and silently reverted to the vmanomaly default.

• BUGFIX: vmanomaly: validate online models' decay against the (0, 1] range while still allowing it to be omitted.

• BUGFIX: vmanomaly: accept a float value (for example 0.01) for the isolation forest contamination parameter in addition to the string auto.

• BUGFIX: vmanomaly: omit the OnlineQuantileModel min_subseason key when it is unset instead of emitting an empty string.

• BUGFIX: vmanomaly: remove the artificial upper bound on spec.server.maxConcurrentTasks; vmanomaly accepts any positive integer.

Update note: -eula flag is not set by default anymore for VMBackup and VMRestore. To avoid VMCluster/VMSingle rollouts set spec.vmstorage.vmBackup.acceptEula: true for VMCluster and spec.vmBackup.acceptEula: true for VMSingle and replace it with spec.license during VMSingle/VMCluster upgrade.

• Dependency: vmoperator: Updated default versions for VM apps to v1.144.0 version

• SECURITY: upgrade Go builder from Go1.25.8 to Go1.25.10. See the list of issues addressed in Go1.25.10.

• BUGFIX: vmagent: use volume from spec.volumes as persistent queue volume if its name is persistent-queue-data, previously emptyDir was mounted. See #1677.

• BUGFIX: vmcluster: use volume from spec.vmstorage.volumes and spec.vmselect.volumes as data and cache volumes if its name is vmstorage-db and vmselect-cachedir respectively. See #784.

• BUGFIX: vmoperator: Improve reconcile error handling for Prometheus and VictoriaMetrics controllers.

• BUGFIX: vmoperator: Add acceptEula support for VMBackup/VMRestore.

• BUGFIX: vmdistributed: change default load balancing policy for write requests from first_available to least_loaded. This should allow to evenly distribute write load across all VMAgents.

• BUGFIX: VMCluster, VTCluster and VLCluster: fixed infinite non-default additional service recreation, when requestsLoadBalancer.enabled: true

• BUGFIX: vmoperator: retry reconcile errors, that may lead to expanding state, before resource could hang in expanding state.

• BUGFIX: vmdistributed: expose VMClusterSpec parsing error in status, previously it was just swallowed and led to infinite reconciles. See #2113.

• BUGFIX: vmanomaly and vmagent: Fix incorrect scaling logs for sharded vmagent and vmanomaly.

• FEATURE: vmauth: support HPA for requests load balancer.

• BUGFIX: vmagent: skip replica count update when VMAgent is in stateful mode and HPA is enabled. See #2190.

• BUGFIX: vmalertmanagerconfig: Remove prefix from empty subroute receiver. See #2185.

• BUGFIX: vmoperator: updated OLM configuration to fix OpenShift catalog publishing. See #2195.

• SECURITY: upgrade Go builder from Go1.26.2 to Go1.26.3. See the list of issues addressed in Go1.26.3.

• Dependency: vmoperator: Updated default versions for VM apps to v1.143.0 version

• Dependency: vmoperator: Updated default versions for VM apps to v1.142.0 version

• FEATURE: vmauth: previously VMAuth could read configuration only from predefined locations; now VMAuth supports arbitrary filesystem access configuration, allowing users to reference required files directly and reducing configuration workarounds. See #899.

• FEATURE: vmuser: support VMAnomaly CRD in VMUser targetRefs. See #2141.

• FEATURE: vmdistributed: introduce spec.zones[*].trafficMode property, which allows disable read, write or whole traffic to a zone. See #1995.

• FEATURE: vmagent: support per remote write queues configuration. See #2138.

• BUGFIX: converter: disable all prometheus controllers if CRD group was not found. See #2838.

• BUGFIX: vmdistributed: change default load balancing policy for write requests from first_available to least_loaded. This should allow to evenly distribute write load across all VMAgents.

• BUGFIX: vmalertmanagerconfig: fix previously ignored negative values in VMAlertmanagerConfig. See #2132.

• BUGFIX: vmalertmanager: fixed ignored alertmanager template if it has no discovered VMAlertmanagerconfig CRs or traci ng config defined. See #2121.

• BUGFIX: VMCluster, VTCluster and VLCluster: fixed infinite non-default additional service recreation, when requestsLoadBalancer.enabled: true

• BUGFIX: vmauth: fixed case, when target_path_suffix can be appended multiple times.

• BUGFIX: vmoperator: retry reconcile errors, that may lead to expanding state, before resource could hang in expanding state.

• BUGFIX: vmcluster, vlcluster and vtcluster: when storage HPA was enabled, generated -storageNode flags could become incorrect after scaling, which could break expected routing to storage nodes; now the operator derives storage node count from the current StatefulSet state so generated flags stay correct during HPA-driven scaling. See #2117.

• BUGFIX: vmdistributed: expose VMClusterSpec parsing error in status, previously it was just swallowed and led to infin ite reconciles. See #2113.

• Dependency: vmoperator: Updated default versions for VM apps to v1.140.0 version

• Dependency: vmoperator: Updated default versions for VM apps to v1.139.0 version Update note 1: deprecated env variables for Prometheus CRs conversion VM_ENABLEDPROMETHEUSCONVERTER_PODMONITOR, VM_ENABLEDPROMETHEUSCONVERTER_SERVICESCRAPE, VM_ENABLEDPROMETHEUSCONVERTER_PROMETHEUSRULE, VM_ENABLEDPROMETHEUSCONVERTER_PROBE, VM_ENABLEDPROMETHEUSCONVERTER_SCRAPECONFIG. Use -controller.disableReconcileFor command-line flag with comma-separated list of controller names, that should be disabled. Update note 2: removed operator_prometheus_converter_watch_events_total metric since migration of Prometheus object watchers to controllers made this counter obsolete. Update note 3: made controller.prometheusCRD.resyncPeriod command line flag noop, which was relevant to Prometheus object watchers. Update note 4: -eula flag is not set by default anymore for VMBackup and VMRestore. To avoid VMCluster/VMSingle rollouts set spec.vmstorage.vmBackup.acceptEula: true for VMCluster and spec.vmBackup.acceptEula: true" for VMSingle and replace it with spec.license during VMSingle/VMCluster upgrade. Update note 5: new flag was introduced, so VMAnomaly pods will be recreated. VMAnomaly version should be at least 1.25. Update note 6: fixed port name collision in VMSingle and VMCluster, when VMBackup is enabled. To avoid VMSingle/VMCluster pods recreation set VM_USE_OLD_BACKUP_RESTORE_PORT_NAMES environment variable to true. Update note 7: the default VMDistributed unauthorizedUserAccessSpec was removed to allow users to decide whether access should be anonymous or authorized. Please check documentation with examples for unauthorized and authorized access and update your CRs accordingly.

• Dependency: vmoperator: Updated default versions for VM apps to v1.139.0 version

• Dependency: vmoperator: Updated default versions for VL apps to v1.50.0.

• FEATURE: helm-converter: new CLI tool that helps with migration from Helm charts to their corresponding Operator Custom Resources (CRs).

• FEATURE: vmsingle: VMSingle reuses vmagent implementation to allow scraping and relabelling. See #1694

• FEATURE: vmoperator: perform statefulset pods deletion instead of eviction when maxUnavailable set to 100%, which is important for minimum downtime strategy. See #1706.

• FEATURE: vmuser: support referencing multiple targets of the same kind at targetRefs[*].crd.objects.

• FEATURE: vmoperator: prettify reconcile diff in logs, now diff objects show only changed JSON fields.

• FEATURE: VLCluster, VLSingle, VLAgent, VTCluster, VTSingle, VMCluster, VMAgent, VMAnomaly, VMAlert, VMAlertmanager, VMAuth: add spec.componentVersion as an alternative to spec.clusterVersion. This field also available in all objects deploying pods. See this #1949 issue for details.

• FEATURE: vmanomaly: add support for settings.retention configuration (ttl and check_interval) in configRawYaml and configSecret. See these docs for details.

• FEATURE: vmuser: support JWT-based auth.

• FEATURE: vmagent: support HPA in VMAgent CR and in VMAgent, which is a part of VMDistributed. See #1961.

• FEATURE: vmagent: VMAgent CRs running in statefulSet mode, including VMAgent components in VMDistributed, now support configuring rolling update strategy behavior. See #1987.

• FEATURE: vmagent: VMAgent CRs running in DaemonSet mode now support configuring rolling update strategy behavior.

• FEATURE: vmoperator: Dry-run mode. See #1832.

• FEATURE: vmanomaly: introduce VMAnomalyConfig CRD to enable dynamic configuration and hot-reload support starting from VMAnomaly version 1.25.0.

• FEATURE: vmalertmanager: introduce arbitrary fs access feature for VMAlertmanager. See #899

• FEATURE: vmalertmanagerconfig: add update_message field to SlackConfig. This allows alertmanager to edit the original Slack message in-place when alert status changes instead of sending a new one. Requires alertmanager v0.32.0+. See #2064.

• BUGFIX: vmbackupmanager: remove deprecated -eula flag from vmbackupmanager and vmrestore container args. See #1319.

• BUGFIX: vmoperator: VMPodScrape for VLAgent and VMAgent now uses the correct port; previously it used the wrong port and could cause scrape failures. See #1887.

• BUGFIX: vmdistributed: updated VMAuth config consolidating all VMSelects into a single read and all VMClusters into a single write backend

• BUGFIX: vmdistributed: fix PVC being owned by StatefulSet and top-level object simultaneously. See #1845.

• BUGFIX: vmoperator: remove unneeded finalizer from core K8s resources. See #835.

• BUGFIX: vmdistributed: remove finalizers from VMServiceScrape and VMPodScrape objects, and keep finalizers on VMAgent, VMCluster, and VMAuth when DeletionTimestamp is not empty.

• BUGFIX: vmsingle and vmagent: previously, ingest-only mode could still mount scrape configuration secrets when relabeling or stream aggregation was configured, which caused unexpected secret mounts and RBAC-related failures; now these secrets are not mounted in ingest-only mode, so deployments start with the expected minimal permissions and avoid related runtime errors. See #1828.

• BUGFIX: vmoperator: recreate STS if immutable fields changed.

• BUGFIX: vmoperator: wait for STS deletion in case of recreation without throwing an error.

• BUGFIX: vmdistributed: ignore VMAuth update/delete operations if controller is disabled.

• BUGFIX: vmalertmanager: fixed ignored tracing config, when no alertmanagerconfig CRs collected. See #1983.

• BUGFIX: vmagent: apply scrape class relabellings before job ones. See #1997.

• BUGFIX: vmanomaly and vmagent: render %SHARD_NUM% placeholder when shard count is greater than 0. See [#2001](https://github.com/VictoriaMetrics/operator/issu

…(truncated)