Sure! It's mainly a mix of APIs to discover publicly available conversations.
Most of the data currently comes from Reddit, particularly as election of subreddits i curated myself. I also collect data from Hacker News, LinkedIn, X, and a few other sources using third party API.
The dataset likely has a bias toward tech, partly because of the subreddits I monitor. I'm working on expanding coverage to make the rankings more representative.
Hi, do you mind sharing how do you fetch or scrape these data
Sure! It's mainly a mix of APIs to discover publicly available conversations.
Most of the data currently comes from Reddit, particularly as election of subreddits i curated myself. I also collect data from Hacker News, LinkedIn, X, and a few other sources using third party API.
The dataset likely has a bias toward tech, partly because of the subreddits I monitor. I'm working on expanding coverage to make the rankings more representative.