Creating index policies for the custom index patterns

Last Updated : May 14, 2025 |

About this task

Set up storage policies for the created indices. The retention values need to be adjusted according to the solution requirements and the storage size.

The fields, "min_index_age" and "min_size" under transition conditions for “read_only” and “delete” need to be configured.

Procedure

  1. On the OpenSearch UI, click Navigation_pane_button to display the navigation pane.
  2. On the navigation pane, go to OpenSearch Plugins > Index Management and click Create policy and then select JSON editor.
  3. In Policy ID, enter a name for the policy, for example: "breeze-default-policy-filebeat."
  4. Replace the contents in Define Policy with the relevant files for each index pattern.

    Example of a sample for "filebeat" JSON policy set for 100 agents lab with PVC Size: 80GB as below:

    {
    "policy": {
        "description": "Adding log retention policy for Breeze logs coming from filebeat.",
        "error_notification": null,
        "default_state": "read_write",
        "states": [
            {
                "name": "read_write",
                "actions": [
                    {
                        "retry": {
                            "count": 3,
                            "backoff": "exponential",
                            "delay": "1m"
                        },
                        "read_write": {}
                    }
                ],
                "transitions": [
                    {
                        "state_name": "read_only",
                        "conditions": {
                            "min_index_age": "2d"
                        }
                    },
                    {
                        "state_name": "read_only",
                        "conditions": {
                            "min_size": "1gb"
                        }
                    }
                ]
            },
            {
                "name": "read_only",
                "actions": [
                    {
                        "retry": {
                            "count": 3,
                            "backoff": "exponential",
                            "delay": "1m"
                        },
                        "read_only": {}
                    }
                ],
                "transitions": [
                    {
                        "state_name": "delete",
                        "conditions": {
                            "min_index_age": "7d"
                        }
                    }
                ]
            },
            {
                "name": "delete",
                "actions": [
                    {
                        "retry": {
                            "count": 3,
                            "backoff": "exponential",
                            "delay": "1m"
                        },
                        "delete": {}
                    }
                ],
                "transitions": []
            }
        ],
        "ism_template": [
            {
                "index_patterns": [
                    "filebeat-*"
                ],
                "priority": 0
            }
        ]
    }
}

    For policy recommendations for different deployment sizes, see Policy recommendations for different deployment sizes.