STUFF

MSA.IDX

The initial idea behind this experiment was to play around with sentiment analysis (estimating the emotional sentiment of text content). Previous tests showed that a( Google is smart at crawling web pages and that b) the tech stack (Remix/React living in a Node server function on AWS) has good performance.

Reddit was chosen as the source for the texts. Two different indexes were created to bring something new (and hopefully useful) to the web. On top of that, a rest API so that users can programmatically access the index data.

A feature is that each query creates a new data point that is stored in the database. In this way, new content is constantly generated and current sentiment values can be compared with values from the past.

There are a few web tools that provide sentiment analysis of Reddit. Mainly used for marketing purposes (brand monitoring), relatively expensive. Offering the service for free and adding some branding should increase the probability that people start using it and it gains popularity over time.

Another feature is the distinction between queries made by humans and those made by bots of search engines or LLMs. - I'm curious to see if machines and humans are interested in the same topics.

While this project is far from perfect, the codebase and concept could serve as a foundation for a commercial project in the future.

Summary
URLmsaidx.com
TypeSearch site, statistical sentiment data
Domain historyNew domain
TimeNovember 2023 - present
StatusBroken due to changes in the API.
Learnings
  • As expected, Google explores and indexes pages and creates new index combinations.