Project Overview
Marketing
- Software Development
- Cloud Solutions
- Solution Architecture
- Data & Analytics
- Quality Engineering
- Laravel
- MySQL
- Vue.js
- Zoho CRM
- Google Analytics
- Facebook Pixel
Requirements
NetReputation.com required an application that could discover and suppress negative results from several sites based on one or multiple keywords. They will offer a service to create articles, reviews, blog posts, etc. with positive content to suppress those negative results by pushing them down in the search results.
Challenges & Solutions
The most difficult challenge in this project was deciding to either crawl and parse all the pages, or use APIs of the different platforms/websites/services to find negative results based on certain keywords. It then needed to be stored in a unified format and regularly updated. This required intensive processing and a complex algorithm.
Google and Bing search APIs were used intensively to ensure no search results were missed. We used a hybrid database approach, i.e. MySQL for the relational data and MongoDB for search results and any other high volume data points.
Create monitoring and alerting in the case a website crawl was not returning results due to structural changes on that website, as crawling and parsing are heavily dependent on HTML structure.
Several monitoring tools like Pingdom and AWS CloudWatch were implemented to ensure the client got alerts in case any service goes down or does not return data.
Due to the large amount of crawling and processing the application needed to do, the right technologies were required to ensure there were no performance issues.
We decided to use Laravel as the backend framework and VueJS for the frontend. As for the hosting, we chose AWS to ensure the application is scalable and could handle heavy computing challenges.
Results
As a result, we built a highly scalable, sustainable, and reliable application for NetReputation.com. The web application could handle high computing needs while crawling websites or fetching data. The application could also handle high data volume due to the hybrid database architecture.
The search results algorithm was tested thoroughly on various data sets to make sure the outcomes were accurate. Users could now compare their initial web appearance vs. current web appearance after buying a suppression service within the application. Users could also see various analytical reports to determine how their web appearance is improving over time.