Configure Index Partition

For indexing, MongoDB Search counts each document as a single index object when it isn't nested inside another document. For embedded documents, MongoDB Search counts each embedded document as additional index objects depending on the number of levels of nesting. MongoDB Search stops replicating changes for indexes larger than 2,100,000,000 index objects.

If you deployed MongoDB Search on separate search nodes, you can increase the number of MongoDB Search index objects by partitioning your index. By default, MongoDB Search supports one partition per shard. Each partition supports up to 2 billion index objects. You can create an index with up to four (4) partitions by using the numPartitions option. These partitions represent a single index with support for up to 8B documents per cluster or shard.

When you configure partitions for your index, MongoDB Search automatically distributes the index objects between the partitions in an optimal way. When you run queries against a collection with index partitions, MongoDB Search scatters the queries to all the partitions and gathers the search results and metadata to sort, merge, and return the results.

We recommend partitioning your index when:

Your index objects reach 50% of the total limit.
The number of documents in your collection reaches two billion.
Your index will contain up to eight billion documents.
Your index is in the STALE state because MongoDB Search stopped replication.

When you configure partitions or modify the number of partitions, MongoDB Search triggers a rebuild of your index.

If you have more than one partition in your cluster, you can't remove all the search nodes and migrate to a deployment model where both the mongod and mongot processes run on the same node.

Syntax

1 {
2   "numPartitions": <integer>
3 }

Supported Values

The MongoDB Search numPartitions option takes the following values:

1 - to create a single index, with no additional partitions. This is the default value.
2 - to create up to two partitions for up to four billion documents.
4 - to create up to four partitions for up to eight billion documents.

Example

The following index example uses the sample_mflix.movies collection to demonstrate how to configure up to 4 partitions for the data in the collection. You can use the Visual Editor or the JSON Editor in the Atlas UI and other supported clients to create the index.

➤ Use the Select your language drop-down menu to set the client of the example in this section.

curl --user "{PUBLIC-KEY}:{PRIVATE-KEY}" --digest \
--header "Accept: application/json" \
--header "Content-Type: application/json" \
--include \
--request POST "https://cloud.mongodb.com/api/atlas/v2/groups/{groupId}/clusters/{clusterName}/search/indexes" \
--data '
  {
  "collectionName": "movies",
  "database": "sample_mflix",
  "name": "partitioned_index",
  "type": "search",
  "definition": {
    "analyzer": "lucene.standard",
    "mappings": {
      "dynamic": true,
    },
    "numPartitions": 4,
    "searchAnalyzer": "lucene.standard"
  }
}'

Create a file named indexDef.json similar to the following:
```
{
    "collectionName": "movies",
    "database": "sample_mflix",
    "definition": {
      "mappings": {
       "dynamic": true
      },
    },
    "name": "partitioned_index",
    "numPartitions": 4
  }
```
Run the following command to create the index.
```
atlas deployments search indexes create --file indexDef.json
```

In Atlas, go to the Search & Vector Search page for your cluster.

You can go the MongoDB Search page from the Search & Vector Search option, or the Data Explorer.

If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar.
If it's not already displayed, select your project from the Projects menu in the navigation bar.
In the sidebar, click Search & Vector Search under the Database heading.
If you have no clusters, click Create cluster to create one. To learn more, see Create a Cluster.
If your project has multiple clusters, select the cluster you want to use from the Select cluster dropdown, then click Go to Atlas Search.
The Search & Vector Search page displays.

If it's not already displayed, select the organization that contains your project from the Organizations menu in the navigation bar.
If it's not already displayed, select your project from the Projects menu in the navigation bar.
In the sidebar, click Data Explorer under the Database heading.
Expand the database and select the collection.
Click the Indexes tab for the collection.
The Atlas Search page displays.

Click Create Search Index.

Start your index configuration.

Make the following selections on the page and then click Next.

Search Type	Select the MongoDB Search index type.
Index Name and Data Source	Specify the following information: Index Name: `partitioned_index` Database and Collection: `sample_mflix` database `movies` collection
Configuration Method	For a guided experience, select Visual Editor. To edit the raw index definition, select JSON Editor.

IMPORTANT:

Your MongoDB Search index is named default by default. If you keep this name, then your index will be the default Search index for any MongoDB Search query that does not specify a different index option in its operators. If you are creating multiple indexes, we recommend that you maintain a consistent, descriptive naming convention across your indexes.

Specify the index definition.

Click Refine Your Index.
Toggle Index Partitions to enable it.
Select 4 from the Number of partitions dropdown and click Save Changes.

Replace the default index definition with the following:
```
{
  "mappings": {
    "dynamic": true
  },
  "numPartitions": 4
}
```
Click Next.

Click Create Search Index.

Atlas displays a Toast (brief, non-interactive notification) to let you know your index is building.

Check the status.

The newly created index appears on the Atlas Search tab. While the index is building, the Status field reads Build in Progress. When the index is finished building, the Status field reads Active.

IMPORTANT: Larger collections take longer to index. You will receive an email notification when your index is finished building.

db.movies.createSearchIndex(
    "search-index",
    { mappings: { dynamic: true }, "numPartitions": 4 }
)

using MongoDB.Bson;
using MongoDB.Driver;
// connect to your Atlas deployment
var uri = "<connection-string>";
var client = new MongoClient(uri);
var db = client.GetDatabase("sample_mflix");
var collection = db.GetCollection<BsonDocument>("movies");
// define your MongoDB Search index
var index = new BsonDocument
{
  { "mappings", new BsonDocument
    {
      { "dynamic", true }
    }
  },
  { "numPartitions", 4 }
};
var result = collection.SearchIndexes.CreateOne(index, "partitioned_index");
Console.WriteLine(result);

import com.mongodb.client.MongoClient;
import com.mongodb.client.MongoClients;
import com.mongodb.client.MongoCollection;
import com.mongodb.client.MongoDatabase;
import org.bson.Document;
public class CreateIndex {
    public static void main(String[] args) {
        // connect to your Atlas cluster
        String uri = "<connection-string>";
        try (MongoClient mongoClient = MongoClients.create(uri)) {
            // set namespace
            MongoDatabase database = mongoClient.getDatabase("sample_mflix");
            MongoCollection<Document> collection = database.getCollection("movies");
            Document index = new Document()
                .append("mappings", new Document()
                   .append("dynamic", true)
                )
                .append("numPartitions", 4);
            collection.createSearchIndex("partitioned_index", index);
        }
    }
}

import { MongoClient } from "mongodb";
// connect to your Atlas deployment
const uri =  "<connection-string>";
const client = new MongoClient(uri);
async function run() {
  try {
    const database = client.db("sample_mflix");
    const collection = database.collection("movies");
    
    // define your MongoDB Search index
    const index = {
        name: "partitioned_index",
        definition: {
            /* search index definition fields */
            "mappings": {
                "dynamic": true
            },
            "numPartitions": 4
        }
    }
    // run the helper method
    const result = await collection.createSearchIndex(index);
    console.log(result);
  } finally {
    await client.close();
  }
}
run().catch(console.dir);

from pymongo.mongo_client import MongoClient
from pymongo.operations import SearchIndexModel
def create_index():
    # Connect to your Atlas deployment
    uri = "<connectionString>"
    client = MongoClient(uri)
    # Access your database and collection
    database = client["sample_mflix"]
    collection = database["movies"]
    # Create your index model, then create the search index
    search_index_model = SearchIndexModel(
        definition={
            "mappings": {
                "dynamic": True
            },
            "numPartitions": 4
        },
        name="partitioned_index",
    )
    result = collection.create_search_index(model=search_index_model)
    print(result)

Back

Synonym Mappings

Query Reference

Syntax

Supported Values

Example

In Atlas, go to the .leafygreen-ui-8n27nz{font-style:normal;font-weight:700;}Search & Vector Search page for your cluster.

Click Create Search Index.

Start your index configuration.

Specify the index definition.

Click Create Search Index.

Check the status.

In Atlas, go to the Search & Vector Search page for your cluster.