hn-recommendation-api
所属分类：数学计算
开发工具：TypeScript
文件大小：0KB
下载次数：0
上传日期：2023-06-23 16:11:26
上 传 者：
sh-1993
说明： 黑客新闻推荐系统。获取给定URL的最相似帖子，
(A recommendation system for Hacker News. Get the most similar posts for a given URL,)
文件列表:
.dockerignore (1124, 2023-06-23)
Dockerfile (522, 2023-06-23)
createIndex.ipynb (5770, 2023-06-23)
database.py (1653, 2023-06-23)
embeddings.py (4865, 2023-06-23)
environment.yml (968, 2023-06-23)
faiss_index.py (576, 2023-06-23)
main.py (2223, 2023-06-23)
next/ (0, 2023-06-23)
next/.dockerignore (368, 2023-06-23)
next/.eslintrc.json (40, 2023-06-23)
next/Dockerfile (202, 2023-06-23)
next/components/ (0, 2023-06-23)
next/components/posts.tsx (1617, 2023-06-23)
next/components/skeleton.tsx (663, 2023-06-23)
next/next.config.js (270, 2023-06-23)
next/package.json (572, 2023-06-23)
next/pages/ (0, 2023-06-23)
next/pages/_app.tsx (1789, 2023-06-23)
next/pages/_document.tsx (231, 2023-06-23)
next/pages/index.tsx (12513, 2023-06-23)
next/pnpm-lock.yaml (90926, 2023-06-23)
next/postcss.config.js (82, 2023-06-23)
next/public/ (0, 2023-06-23)
next/public/favicon.png (8153, 2023-06-23)
next/public/og-image.png (61558, 2023-06-23)
next/styles/ (0, 2023-06-23)
next/styles/globals.css (280, 2023-06-23)
next/tailwind.config.js (468, 2023-06-23)
next/tsconfig.json (594, 2023-06-23)
# hn-recommendation-api
This project provides a recommendation engine for Hacker News posts based on their content. It utilizes the concept of embeddings to compute the similarity between posts. Specifically, the text of each post is extracted using the Diffbot API, and embeddings are computed using the OpenAI API with the "text-embedding-ada-002" model. An HNSW index is built using Faiss to quickly find the nearest items.
## Components
The project consists of three parts:
- `root`: The API using Python and FastAPI.
- `next`: The website using Next.JS and Typescript.
- `createIndex.piynb`: The Jupyter notebook to convert the dataset to a Faiss HNSW index.
## Dataset
The dataset used to train the embeddings is available on Kaggle: https://www.kaggle.com/datasets/julien040/hacker-news-openai-embeddings
## Blog Post
A blog post about extracting the embeddings of Hacker News posts to get a recommendation engine is available at https://julienc.me/articles/Extract_embeddings_Hacker_News_article
## Contact
For any questions or API access requests, please contact me at contact[at]julienc.me.
## Disclaimer
This project is not affiliated with Y Combinator or Hacker News.
## License
This project is licensed under the MIT License.
近期下载者：
相关文件：
收藏者：