Recovering ISM policy detachment after template updates

After an index template is edited — a mapping tweak, a shard-count change, a settings cleanup — the very next rollover can quietly produce an unmanaged index: it comes up with no OpenSearch Index State Management (ISM) policy attached, never transitions, and grows without bound while every older index in the series keeps tiering normally. The usual cause is that the template edit dropped the plugins.index_state_management.policy_id setting (or the policy’s ism_template no longer matches the pattern), so ISM has nothing to auto-attach at index creation. This procedure detects the unmanaged indices, re-attaches the policy with _plugins/_ism/add, and repairs the template so future rollovers self-manage again.

The trap is that nothing fails loudly: the template update succeeds, the rollover succeeds, and the write path keeps serving. Only the lifecycle silently stops for new indices, so you discover it days later when a hot node fills. Detaching a policy at creation time is a Phase Transition Logic failure — the transitions never start — and re-attaching a whole generation of indices at once is exactly the job of Bulk Policy Attachment. Getting the template’s rollover wiring right in the first place is covered under Rollover Trigger Configuration.

Prerequisites #

Confirm each item before re-attaching — a bulk _ism/add against indices that already have a policy silently no-ops, and re-attaching without fixing the template just recreates the problem on the next roll.

• You can call _plugins/_ism/explain, _plugins/_ism/add, and read/update index templates (_index_template).
• You know the policy id that should be attached and the index pattern it applies to, matching the transitions defined in Phase Transition Logic.
• You know whether attachment is driven by the template setting plugins.index_state_management.policy_id or by the policy’s own ism_template block — OpenSearch supports both and they fail differently.
• For a large detached generation, you have the bulk workflow ready from Bulk Policy Attachment so you re-attach in one pass rather than per index.
• The write alias and rollover wiring are intact, per Rollover Trigger Configuration, so a re-attached index can still roll.

Step-by-step procedure #

Detect the full set of unmanaged indices first, re-attach them, then fix the template so the leak stops — doing the template fix alone leaves the already-created indices stranded forever.

1. Detect unmanaged indices in the series #

_plugins/_ism/explain reports a policy_id of null for any index ISM is not managing. Query the whole pattern and let the null values reveal the detached generation.

GET _plugins/_ism/explain/logs-*?pretty

{
  "logs-000003": { "index.plugins.index_state_management.policy_id": "logs-hot-warm-cold", "enabled": true },
  "logs-000004": { "index.plugins.index_state_management.policy_id": null, "enabled": null },
  "logs-000005": { "index.plugins.index_state_management.policy_id": null, "enabled": null },
  "total_managed_indices": 3
}

Gotcha: an unmanaged index shows policy_id: null and enabled: null, not enabled: false — a false means the policy is attached but paused, which is a different problem. Note that total_managed_indices counts only the managed ones, so a count lower than your index total is the fastest smoke test that something detached.

2. Script the detection so you catch every detached index #

For anything past a couple of indices, filter programmatically. This opensearch-py snippet lists exactly the indices that need re-attaching, ignoring the integer summary key.

from opensearchpy import OpenSearch

client = OpenSearch("https://os.internal:9200", http_auth=("svc_ism", "..."), verify_certs=True)

explain = client.transport.perform_request("GET", "/_plugins/_ism/explain/logs-*")
unmanaged = [
    idx for idx, meta in explain.items()
    if isinstance(meta, dict)
    and meta.get("index.plugins.index_state_management.policy_id") is None
]
print(f"{len(unmanaged)} unmanaged indices: {unmanaged}")

2 unmanaged indices: ['logs-000004', 'logs-000005']

Gotcha: guard with isinstance(meta, dict) — total_managed_indices is an integer sitting alongside the index keys, and calling .get() on it raises AttributeError. This is the same key-filtering discipline the bulk workflow relies on.

3. Re-attach the policy with _plugins/_ism/add #

Attach the policy to the detached indices. _ism/add accepts a comma-separated list or the pattern, and it only affects indices that currently have no policy — already-managed indices are left untouched.

POST _plugins/_ism/add/logs-000004,logs-000005
{
  "policy_id": "logs-hot-warm-cold"
}

{
  "updated_indices": 2,
  "failures": false,
  "failed_indices": []
}

Gotcha: _ism/add starts the index at the policy’s default_state, not at whatever state a same-age managed sibling is in — a week-old index that should already be warm will re-enter at hot and re-run those transitions. If that matters, drive it forward with POST _plugins/_ism/change_policy and an explicit target state instead.

4. Find the template edit that dropped the policy setting #

The detached indices are only a symptom; the template is the leak. Inspect the composable template backing the pattern and confirm whether the ISM setting is present.

GET _index_template/logs-template?filter_path=index_templates.index_template.template.settings

{
  "index_templates": [
    { "index_template": { "template": { "settings": { "index": {
      "number_of_shards": "3",
      "number_of_replicas": "1"
    } } } } }
  ]
}

Gotcha: the absence of any plugins.index_state_management.policy_id under settings.index is the smoking gun — an earlier edit replaced the whole settings block and dropped it. If you instead rely on the policy’s ism_template for auto-attach, the setting legitimately won’t be here; check the policy’s ism_template.index_patterns still matches logs-* in that case.

5. Repair the template so future rollovers self-manage #

Put the ISM setting back into the template (or fix the policy’s ism_template pattern). This does not touch existing indices — it only governs indices created from the next rollover onward.

PUT _index_template/logs-template
{
  "index_patterns": ["logs-*"],
  "template": {
    "settings": {
      "index.number_of_shards": 3,
      "index.number_of_replicas": 1,
      "plugins.index_state_management.policy_id": "logs-hot-warm-cold",
      "plugins.index_state_management.rollover_alias": "logs-write"
    }
  },
  "priority": 200
}

{ "acknowledged": true }

Gotcha: re-adding policy_id also requires the rollover_alias setting to survive the edit, or the re-managed index rolls but the new backing index is unmanaged again — the two settings must travel together. Bump priority above any conflicting template so yours wins the match.

Verification #

Confirm every index is managed and the next-created index inherits the policy.

# 1. Are the previously detached indices now managed?
curl -s "https://os.internal:9200/_plugins/_ism/explain/logs-000004?pretty"
# "index.plugins.index_state_management.policy_id": "logs-hot-warm-cold", "enabled": true

# 2. Does the total managed count now equal the index count?
curl -s "https://os.internal:9200/_plugins/_ism/explain/logs-*?filter_path=total_managed_indices"
# { "total_managed_indices": 5 }

# 3. Does a freshly rolled index inherit the policy from the fixed template?
curl -s -XPOST "https://os.internal:9200/logs-write/_rollover" >/dev/null
curl -s "https://os.internal:9200/_plugins/_ism/explain/logs-000006?filter_path=**.policy_id"
# "index.plugins.index_state_management.policy_id": "logs-hot-warm-cold"

A healthy result shows total_managed_indices equal to the number of backing indices, every index reporting a non-null policy_id, and a newly rolled logs-000006 inheriting the policy automatically. If the freshly rolled index still comes up null, the template fix did not take — recheck priority and that the pattern matches.

Common failures #

Frequently asked questions #

Related #

• Bulk Policy Attachment — re-attaching a large detached generation in a single scaled pass.
• Rollover Trigger Configuration — keeping policy_id and rollover_alias wired together in the template.
• Phase Transition Logic — the transitions a detached index never gets to run.

Up: Phase Transition Logic