Scaling Search Capability Using AWS

Introduction

Xtendlabs is a PaaS-based company offering clients data science and big data engineering services. In their portfolio, they have clients from a variety of domains.

Case Study

we delivered the value by making the search capability faster, more reliable, and at scale using keywords via Elastic AppSearch.

What are the Challenges faced by the Customer?

The data came in the form of Excel files where every file has a minimum of 44K records and has 47 attributes that contain the details of the companies like company name, location, domain, number of employees, revenue, etc. Initially, it was stored in Google Drive and caused increased latency while dumping the data to Elastic AppSearch.

Consequences

Increase in waiting time: Since the writing, the job was taking a long time the customer needed to wait to perform a search on Elastic AppSearch.
Frequent job timeouts: The dumping job was running on cloud infrastructure because of the high latency the job used to time out.
Security: To read the data from Google Drive, the link was given public access, i.e., the data can be misused if the link is in the wrong hands.