Add full-text search to your Firebase app

As it is publicly known, Firestore as a database does not allow full-text searches. So in case you would like to run a query with an operator such as LIKE (as you would do in a SQL database), you would hit a wall. It is possible however to make such a query, but only integrating third-party software as a sort of search engine. The Firebase team recommends using Algolia, but I find it extremely expensive for simple projects (although it provides several client libraries to integrate it seamlessly with your apps). In this tutorial, we are going to use another very popular alternative, which is Elasticsearch

In this article, we are going to add a search functionality to the Firebase-based REST API we developed in this previous article

Repo for this article is available here

Elasticsearch

As described in the What is Elasticsearch page, it is a distributed, open source search and analytics engine for all types of data, including textual, numerical, geospatial, structured, and unstructured [...]. Sounds pretty complex and I don't certainly know what it means, but I do know it can help us with our use case.

You can use Elasticsearch's JavaScript client both from your web app (client-side code), or from your server (node.js code). As we are building a REST API, we are just going to communicate with Elasticsearch from our server.

First of all, we need an instance of Elasticsearch running. We can rely on a cloud provider (such as elastic.co or bonsai.io), or spin our own instance locally using Docker (this is an incredible advantage over Algolia, as as far as I know, you cannot run an instance of Algolia locally). Create your docker container with the following command

docker run -d --name elasticsearch -p 9200:9200 -p 9300:9300 -e "discovery.type=single-node" elasticsearch:7.9.3

Version 7.9.3 is the current latest up to this day (Oct 31, 2020)

Then, we need to install Elasticsearch's SDK for JavaScript

npm install @elastic/elasticsearch

And we are done with the set up. We are now ready to index our documents when we do a write to the database (create or update a document) and search them when doing a read (reading from the database).

Indexing data

First of all, create your ES client

import { Client } from "@elastic/elasticsearch"
...

const app = express()
// Change the URL accordingly to your situation
const esClient = new Client({ node: "http://localhost:9200" });
const ES_INDEX_NAME = "contacts";

Then, in your POST handler, index your contact

app.post("/contacts", async (req, res) => {
  // Validate data and create document in here

  await esClient.index({
    index: ES_INDEX_NAME,
    id: doc.id,
    body: { firstName, lastName, address},
  })
  ...
});

You would want to do the exact same thing when updating a contact. Oh and don't forget to remove the document from the index when deleting it from the database! You can achieve that with the following code

await esClient.delete({ index: ES_INDEX_NAME, id: doc.id })

Searching data

Now, the one feature we are trying to achieve with all of these third-party integration - full-text search! We are now going to add support for a query parameter to our list endpoint. If present, we are going to search and return only those contacts that match a certain criteria dictated by the query parameter value. If not present, we are going to return all contacts

app.get("/contacts", async (req, res) => {
  const { q } = req.query;
  if (q) {
    // Search documents in ES
  } else {
    // Return all documents
  }
})

We will use the query parameter to search every document that matches in any of its fields with the query parameter value

const { body } = await esClient.search({
  index: ES_INDEX_NAME,
  body: {
    query: {
      bool: {
        should: [
          { wildcard: { firstName: `*${q}*` } },
          { wildcard: { lastName: `*${q}*` } },
          { wildcard: { address: `*${q}*` } }
        ]
      }
    }
  }
});

const contactIds: string[] = body.hits.hits.map(
  (hit: { _id: string }) => hit._id
);

const contacts = await Promise.all(
  contactIds.map(async id => {
    const doc = await contactsCollection.doc(id).get();
    if (!doc.exists) return;
    return { id: doc.id, ...doc.data() };
  })
);

return res.json({
  data: { contacts: contacts.filter(contact => contact) }
});

That's a lot of code, I know, but the most important part of it is actually the first one, when we are reaching to our Elasticsearch instance to search documents that matches with the query parameter. What we do then is obtain those documents' ids, fetch those documents from the database and return them.

Conclusion

In this article we saw how to implement one of the most requested features for Firestore (which I don't think will be implemented natively, due to how Firestore works internally), which is full-text search. We accomplished this by integrating a third-party software, such as Elasticsearch.

Hope you liked it!

🐦 @LucianoSerruya

📧 lucianoserruya (at) gmail (dot) com