OpenSearch

OpenSearch is what happens when you take a distributed search engine and make it general purpose. It started life as a fork of Elasticsearch 7.10 in 2021, after the licence change, and now sits under the OpenSearch Software Foundation with an Apache 2.0 licence. If you have used Elasticsearch, most of what you know still applies. The APIs are close enough that migrating between them is usually a configuration exercise rather than a rewrite.

What follows is the working knowledge you actually need before putting one into production. Not the feature list, the parts that bite.

The mental model

An OpenSearch cluster holds indices. An index holds documents, which are JSON objects. Each index is split into shards, and each shard is a self contained Lucene index that can live on any node in the cluster. Replicas are copies of shards on other nodes, giving you both redundancy and extra read throughput.

That is the whole architecture. Everything else is detail on top of it. When a query arrives, the coordinating node fans it out to one copy of every shard, each shard produces its own top results, and the coordinator merges them. This is why shard count matters so much: too many shards and every query pays a fan out cost it does not need, too few and you cannot spread the data or the load.

The usual guidance is to aim for shards between 10 and 50 GB, and to keep total shard count per node in the low hundreds at most. A five node cluster with 2,000 tiny shards will be slower than the same cluster with 50 well sized ones.

Mappings, and why dynamic mapping will eventually hurt you

A mapping is the schema for an index. It says which fields exist and how each one is treated. If you do not define one, OpenSearch guesses from the first document it sees. That is convenient for a prototype and a problem in production, because the guess is permanent. You cannot change the type of an existing field. You reindex.

The classic failure looks like this. A field arrives as "12345" in the first document, so it becomes a text field. Two weeks later something sends 12345 as a number, and the whole bulk request fails. Or worse, an id field gets mapped as text, gets analysed into tokens, and exact match lookups quietly stop working.

Define your mappings explicitly, using an index template so new indices pick them up automatically:

PUT _index_template/orders
{
  "index_patterns": ["orders-*"],
  "template": {
    "settings": {
      "number_of_shards": 3,
      "number_of_replicas": 1,
      "refresh_interval": "5s"
    },
    "mappings": {
      "dynamic": "strict",
      "properties": {
        "order_id":    { "type": "keyword" },
        "customer":    { "type": "text", "fields": { "raw": { "type": "keyword" } } },
        "status":      { "type": "keyword" },
        "total":       { "type": "scaled_float", "scaling_factor": 100 },
        "created_at":  { "type": "date" }
      }
    }
  }
}

"dynamic": "strict" makes OpenSearch reject documents with unmapped fields instead of silently inventing types for them. It turns a slow data quality problem into a fast, visible error, which is almost always the better trade.

Text versus keyword

This single distinction causes more confusion than anything else in the system.

A text field is analysed. The string is run through a tokeniser and a chain of filters, broken into terms, lowercased, possibly stemmed, and those terms go into the inverted index. "Blue Widget Ltd" becomes blue, widget, ltd. You can then search for "widget" and find it.

A keyword field is not analysed. The entire string is stored as one term. "Blue Widget Ltd" stays "Blue Widget Ltd", case and all. You cannot search for "widget" and find it, but you can filter on it exactly, sort by it, and aggregate on it.

You usually want both, which is what the multi field in the template above does. customer for searching, customer.raw for filtering and grouping.

The matching rule follows from this. A match query analyses your search string the same way the field was analysed, then looks for the resulting terms. A term query does not analyse anything and looks for the literal value. Running a term query against a text field with a capitalised value is the single most common "why does this return nothing" in OpenSearch, because the index holds widget and you asked for Widget.

Queries: must, should, filter

Real queries are almost always bool queries with several clauses. The four clause types are worth memorising:

• must: has to match, and contributes to the relevance score.

• should: optional, but matching raises the score. Useful for boosting.

• filter: has to match, contributes nothing to the score.

• must_not: has to not match, contributes nothing.

The performance point is that filter and must_not run in a non scoring context, and OpenSearch caches those results. Anything binary belongs in filter. Date ranges, status equality, tenant IDs, permission checks. Putting them in must works, but you pay to score something where the score is meaningless, and you lose the cache.

{
  "query": {
    "bool": {
      "must": [
        { "match": { "description": "wireless keyboard" } }
      ],
      "filter": [
        { "term":  { "status": "active" } },
        { "range": { "created_at": { "gte": "now-90d" } } }
      ],
      "should": [
        { "match_phrase": { "description": { "query": "wireless keyboard", "boost": 3 } } }
      ]
    }
  }
}

The should clause here is a common relevance trick. Documents containing the exact phrase score higher than those merely containing both words somewhere, without excluding the latter.

How relevance is calculated

OpenSearch scores with BM25 by default. Three things drive the number: how often the term appears in the document, how rare the term is across the index, and how long the document is. Rare terms count for more, and a match in a short field counts for more than the same match buried in a long one.

Two consequences catch people out. First, scores are not comparable between queries. A score of 8.4 means nothing on its own, so a global relevance threshold is a bad idea. Second, scores are calculated per shard by default, which means term statistics differ slightly between shards. On a small index with uneven distribution you can get results that look wrong. search_type=dfs_query_then_fetch fixes it at some cost, but usually the right answer is to have enough data that the statistics even out.

Aggregations

Aggregations are the other half of OpenSearch, and the reason it ends up doing analytics work it was not obviously designed for. They run over the documents matching your query, so you get faceted search and dashboards from the same request that returns results.

{
  "size": 0,
  "query": { "term": { "status": "active" } },
  "aggs": {
    "by_month": {
      "date_histogram": { "field": "created_at", "calendar_interval": "month" },
      "aggs": {
        "revenue": { "sum": { "field": "total" } }
      }
    }
  }
}

"size": 0 tells it to skip the documents and return only the aggregation, which is what you want when you are drawing a chart.

Be careful with terms aggregations on high cardinality fields. They are approximate by default, they are computed per shard and merged, and the counts on rare buckets can be wrong. The doc_count_error_upper_bound in the response tells you how wrong.

Vector search

OpenSearch has had k nearest neighbour search for years, and it is now good enough that it is a genuine option for retrieval augmented generation rather than a checkbox.

You declare a knn_vector field and turn on the index setting:

PUT documents
{
  "settings": { "index.knn": true },
  "mappings": {
    "properties": {
      "chunk": { "type": "text" },
      "embedding": {
        "type": "knn_vector",
        "dimension": 1024,
        "method": {
          "name": "hnsw",
          "space_type": "cosinesimil",
          "engine": "faiss",
          "parameters": { "m": 16, "ef_construction": 128 }
        }
      }
    }
  }
}

The engine choice matters. faiss supports both HNSW and IVF, and supports quantisation, so it is the default choice for anything large. lucene keeps everything in the Lucene segment and handles filtered search well. nmslib is deprecated and should not be used for new work.

What makes OpenSearch interesting for vector work is not the vector search itself, it is that you get keyword search in the same engine over the same documents. Hybrid search combines both and normalises the scores, which no amount of clever embedding will match on its own:

PUT _search/pipeline/hybrid-pipeline
{
  "phase_results_processors": [
    {
      "normalization-processor": {
        "normalization": { "technique": "min_max" },
        "combination": {
          "technique": "arithmetic_mean",
          "parameters": { "weights": [0.3, 0.7] }
        }
      }
    }
  ]
}

You then run a hybrid query with a match clause and a knn clause, and the pipeline merges them. The normalisation step is the important bit. BM25 scores and cosine similarities live on completely different scales, so adding them raw gives you nonsense.

Running one

A few things determine whether an OpenSearch cluster is pleasant or painful to operate.

Heap. Set the JVM heap to half of available RAM, and never above about 31 GB. Past that the JVM loses compressed object pointers and you get less usable memory from more heap. The other half of RAM is not wasted, it becomes filesystem cache, which is what makes Lucene fast.

Refresh interval. The default is one second, meaning newly indexed documents become searchable within a second. That is expensive for bulk loading. If you are ingesting a large batch, set refresh_interval to -1, load, then set it back and force a refresh. On log style indices where near real time is not required, 30s is a reasonable steady state.

Index lifecycle. Use Index State Management with data streams for anything time based. Roll over to a new index on size or age, move older indices to cheaper storage, force merge them down to one segment, then delete them on schedule. Doing this by hand with cron and curl works right up until the night it does not.

Snapshots. Register a repository pointing at object storage and take snapshots on a schedule. Snapshots are incremental at segment level, so daily is cheap. Restore is the only real disaster recovery story, so test it before you need it.

Nodes with roles. On anything beyond a small cluster, separate dedicated cluster manager nodes from data nodes. Three cluster manager nodes, no data on them. A cluster manager that gets stalled by a heavy search is a cluster that loses quorum.

Managed or self hosted

AWS offers OpenSearch Service, which handles provisioning, patching, snapshots and version upgrades. You still choose instance types and shard layout, so it removes the operations toil rather than the design work. OpenSearch Serverless goes further and removes capacity planning too, billing on compute units, which suits spiky or unpredictable workloads but costs more at steady high volume.

Self hosting is entirely reasonable, particularly on Kubernetes with the OpenSearch operator, and it is meaningfully cheaper at scale. The question is whether you have someone who will own JVM tuning, version upgrades and the 3am disk pressure alert. If the honest answer is no, take the managed service.

When to use something else

OpenSearch is not a primary datastore. It has no transactions, its consistency model is eventual, and a mapping mistake can mean a full reindex from source. Always keep the authoritative copy somewhere else and treat the index as derived.

If you only need exact filtering and simple text matching over modest data, Postgres full text search will do the job with one fewer system to run. If you need vector search and nothing else, a dedicated vector store or the pgvector extension is simpler. OpenSearch earns its place when you need full text relevance, faceted aggregation and vector search over the same corpus, at a size where a single database will not keep up.

Getting it right first time

Most of the pain we see with search platforms comes from decisions made in the first week. Shard counts picked arbitrarily, mappings left dynamic, no lifecycle policy, no plan for reindexing. All of them are cheap to fix on day one and expensive to fix on day 400.

Eight Mile designs, builds and operates search and data platforms, including OpenSearch clusters, hybrid retrieval for AI systems, cloud infrastructure and the backend services around them. If you are standing up a new cluster, or you have one that has stopped behaving, get in touch and we will talk it through.