Built this: Social media sentiment and trend analytics on MongoDB

On 8 October 2026 I committed my Big Data Analytics project: a full pipeline from raw tweets to a dashboard.

What’s in it

From the README:

• Ingestion: streams CSV/JSON/JSONL, validates and normalises, extracts hashtags, mentions, URLs and emoji, de-duplicates, and bulk-inserts in batches.
• MongoDB: a posts collection with $jsonSchema validation, 12 indexes, and 5 rollup collections built with aggregation pipelines.
• NLP: sentiment (TF-IDF + logistic regression, evaluated on TweetEval), topics (TF-IDF + NMF), per-post keywords.
• Analytics: sentiment, topics, trends, engagement, time patterns, geography, language and anomaly detection (rolling z-score, IQR, Isolation Forest).
• API: 26 FastAPI endpoints. Dashboard: React 19 + Vite + Tailwind + Recharts, 11 pages.
• Big Data Lab: a PySpark bronze/silver/gold Parquet data lake, and a 3-shard MongoDB cluster comparing hashed and ranged shard keys.
• Tests: 40 backend tests plus frontend unit tests.

The data, as the README describes it

• 2,944,814 tweets from FiveThirtyEight’s russian-troll-tweets (Linvill and Warren, Clemson University, CC BY 4.0), dated 2012 to 2018. They come from accounts linked to the Internet Research Agency, so the results describe that campaign, not ordinary users.
• Engagement counts are synthetic and flagged as such everywhere.
• Sentiment is a model prediction trained on TweetEval (test macro-F1 0.58) and was not evaluated on these tweets themselves.

Where it connects

The sentiment and TF-IDF parts come straight from my sentiment analysis and TF-IDF notes, now at a scale of millions of documents.