A unique web app that watches the real estate market in real time

Visidoo makes life easier for anyone looking for a place to live. People chasing their dream property no longer need to trawl real estate listing sites every day to check for new offers. All they have to do is enter their dream property's criteria on the Visidoo website and leave their contact details. Every minute, Visidoo scans online real estate listings across Czechia, and the moment it finds a property matching what the client is looking for, it lets them know instantly.

I know what it's like to have your dream house or apartment slip away. I've missed out on a great opportunity myself, more than once, and it still bothers me to this day. That's why we founded Visidoo. We came to Superkoders with an idea for how property searches could be automated, and they turned that idea into reality.

Visidoo's founders came to us with a groundbreaking idea for an app that would regularly monitor real estate listing sites and automatically alert users the moment a relevant listing matching their saved criteria appeared on the market. Together, we then came up with the concept of the ultimate watchdog covering every major real estate site in the Czech Republic.

Visidoo monitors real estate sites every minute (with a few minor exceptions — for instance, a longer interval is set overnight). When it finds a new listing, it compares it against users' saved search criteria and immediately sends an email notification to anyone whose criteria match the listing. That lets interested buyers respond to a new listing within minutes — often to the great surprise of real estate agents.

How does it actually work? We built the Visidoo app so that it scrapes real estate sites. Visidoo goes through real estate sites, scrapes the data it needs from them, and compares it against the criteria users have entered into the system. This process repeats every minute across dozens of different real estate sites — and our job was to program the whole thing so it would run automatically and without a hitch. And with serious throughput — Visidoo processes roughly 100,000 real estate listings a month.

The technically toughest part of building the app was the sheer volume of data Visidoo works with. We had to find a way to read and write data with wildly varying structures, sort orders, and filter settings — and sometimes even different terminology. A typical example is 'floor' versus 'storey.'

During testing the app, we tackled two big challenges — filtering out duplicate listings for the same property from different real estate sites, and situations where someone posts a property listing to a real estate site incorrectly or outside the established standard. We ran into plenty of complications an ordinary person would never notice. For example, when someone adds a vertical bar to a listing title — to human eyes, that just looks like a nicely structured title, but for the app, it was an insurmountable obstacle.

That's what led us to the idea of bringing in artificial intelligence:

Bringing in artificial intelligence solves both the duplicate-listing problem and the issue of people filling in listings incorrectly. It can read real estate websites on its own, understand context the way a person would, and isn't thrown off by an odd extra character. We currently have the AI version of Visidoo in the integration phase. It looks like, going forward, we won't need unique code for every property listing site anymore, and we won't have to keep adapting and tweaking that code as real estate sites keep changing in their own ways.

• The Visidoo web app runs on the serverless technology AWS Lambda.
• The app can scrape both the frontend and backend of real estate sites.
• We store the scraped data in Elasticsearch.
• For the programming language, we chose TypeScript.
• We're currently testing AI integration.

Thanks to AWS Lambda, Visidoo doesn't need servers or clusters. That means if we need to scale up the app's performance, we don't have to worry about whether a server can handle it. Serverless technology is also very easy to scale — we can easily turn it from a system handling a handful of requests a day into one that can process thousands of events per second. That's what lets us check dozens of real estate sites every minute.

Also easy to scale is Elasticsearch, which we chose because it's fast and offers high performance for both reads and writes. We like the programming language TypeScript (a superset of JavaScript) because it helps us prevent bugs in the code.

A fun fact for the tech-curious: it's easier for developers to code scraping for the backend, since it's always more structured and therefore easier to extract data from. Unfortunately, not every site has a publicly accessible backend.

For the Visidoo app, we also built

• integration with third-party APIs,
• we set up the Visidoo API so that business partners,
• custom email templates,
• advanced analytics tools that let the business monetize and scale.

We built the Visidoo web app in record time. From the first idea to a finished, working tool, only about 8 months.

• March 2021 — first idea
• August 2021 — working version (MVP)
• September 2021 — testing
• November 2021 — launch

For Visidoo, agile development proved to be the right call. We came on board right at the start, when the project was still in the hypothesis-testing phase. We worked through those hypotheses together with the client, running in weekly sprints.

Our collaboration with Visidoo isn't over — we have plenty of shared plans ahead. In line with current trends, we're looking for the best ways to use AI for the project; we're moving to a premium version of the project and are also planning a marketplace with affiliated services. A separate challenge is the Visidoo mobile app, which is currently in internal testing.