Data Aggregation Platform for AC Installation Companies
Our client’s goal was to create a centralized platform listing all air conditioning installation companies in Germany. The main task was to collect accurate and comprehensive information from various online sources, where the data was often scattered, unstructured and difficult to access.
We used Webflow low-code technology to create a visually appealing and user-friendly platform with responsive design features that ensures smooth navigation on different devices. To keep the information on the website up-to-date, we implemented the logic of regular updating of the entire data set after a certain period of time, without disruptions in the website’s operation.
Challenges
- Data Fragmentation: Installers' information was scattered across multiple sources, requiring a way to efficiently aggregate data into a unified format.
- Search Optimization: Finding relevant businesses required leveraging the Google Places API to accurately locate air conditioning installers across Germany. However, retrieving detailed data from their websites presented an additional layer of complexity.
- Dynamic Website Scraping: Extracting specific details such as addresses, phone numbers, and emails required automating search interactions on websites using Puppeteer to navigate and scrape dynamic content effectively.
- Anti-Scraping Measures: Overcoming anti-scraping mechanisms on some websites necessitated the implementation of adaptive techniques to avoid detection and ensure consistent data retrieval.
- Data Validation: Ensuring the accuracy and consistency of scraped data while minimizing manual verification was critical for maintaining the directory's reliability.
Node JS
Webflow
Puppeteer
Amazon EC2
Solutions & Technologies
Webflow Integration
Webflow as a low-code solution allowed us to significantly reduce website development time while maintaining flexibility in content management. The platform’s integrated CMS ensured easy and automated updating of the catalog of air conditioning companies without the need for constant manual intervention.
Thanks to dynamic collections and flexible filters, users can conveniently browse, search for companies and sort results by the desired criteria.
Results
We have successfully automated the process of filling the site with a large amount of data, which allows the client to save time and resources. During the first launch of the scraper, more than 1500 pages were created with profiles of companies that install air conditioners in Germany.
Thanks to the effective integration of the scraper with Webflow, the site demonstrates high stability even when a large amount of data is updated simultaneously. Thanks to the smart selection of technologies, the client’s costs were minimized: they only pay for a Webflow subscription and scraper hosting on AWS.
Other Cases
