文件列表:

.env.template (152, 2023-02-06)
.eslintrc.json (40, 2023-02-06)
jsconfig.json (93, 2023-02-06)
next.config.js (118, 2023-02-06)
package-lock.json (250734, 2023-02-06)
package.json (646, 2023-02-06)
pages (0, 2023-02-06)
pages\_app.js (145, 2023-02-06)
pages\_document.js (231, 2023-02-06)
pages\api (0, 2023-02-06)
pages\api\help.js (3570, 2023-02-06)
pages\index.jsx (4457, 2023-02-06)
preview.png (75872, 2023-02-06)
public (0, 2023-02-06)
public\favicon.ico (25931, 2023-02-06)
public\next.svg (1375, 2023-02-06)
public\thirteen.svg (1138, 2023-02-06)
public\vercel.svg (629, 2023-02-06)
sitemap.json (70, 2023-02-06)
tasks (0, 2023-02-06)
tasks\1-sitemap-to-csv.js (0, 2023-02-06)
tasks\2-scrape.js (1998, 2023-02-06)
tasks\3-generate-embeddings.js (2695, 2023-02-06)
tasks\package.json (334, 2023-02-06)

**This is a research demo. Support is not provided. # Shopify Help Center Search via GPT Quickly surface answers from Shopify's help center using GPT. ## Technologies used - ScrapingBee to scrape list of help center urls - Mongodb to store scraped data - OpenAI to created embeddings vector points and completion prompt - Pinecone to store vectors in db ## How this works 1. Run `tasks/1-sitemap-to-csv.js` to convert Shopify's Help Center Sitemap.xml into CSV and drop all columns except urls. 2. Convert CSV into array of links. 3. Run `tasks/2-scrape.js` to scrape the article text from every link using ScrapingBee and isnert text into Mongodb using url as unique index. 4. Run `tasks/3-generate-embeddings.js` to generate OpenAI embeddings and upsert into Pinecone. ## Why? Created this as a research experiment in order to learn OpenAI embeddings + Pinecone. Added bonus was to have a way to quickly surface answers for my Shopify platform questions. ## How to optimize this further - Split article text into smaller documents to decrease cost of token usage. Split by H2/section. - Test different models to see one cost. Curie is 10x cheaper than Davinci. - Search documents with a normal search engine (Algolia) and pass that document into open AI rather than using embeddings and Pinecone. - Cache results for common queries. - Test a shorter prompt to further save tokens. ## Preview 

近期下载者：

相关文件：

评论：[我要评论] [举报此文件]

收藏者：