Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Research with local documents #452

Merged
merged 67 commits into from
May 20, 2024

Conversation

ElishaKay
Copy link
Contributor

@ElishaKay ElishaKay commented Apr 25, 2024

Fixes an error when running on mac:

docker-compose up --build

The error (both on this branch & master branch) is:

Failed to fetch http://deb.debian.org/debian-security/pool/updates/main/f/firefox-esr/firefox-esr_115.3.1esr-1%7edeb11u1_arm64.deb 404 Not Found [IP: 199.232.82.132 80]
#0 2.622 E: Unable to fetch some archives, maybe run apt-get update or try with --fix-missing?

@assafelovic
Copy link
Owner

Thanks @ElishaKay ! Did you see that there are many conflicts?

@ElishaKay ElishaKay force-pushed the document-reports branch 2 times, most recently from 108cdc2 to 7d67727 Compare April 29, 2024 17:56
@assafelovic assafelovic changed the title Document reports branch - with Dockerfile fix for mac Research with local documents May 2, 2024
@arsaboo
Copy link
Contributor

arsaboo commented May 5, 2024

Don't we also need to modify the docker-compose to provide the mount path?

@ElishaKay
Copy link
Contributor Author

ElishaKay commented May 10, 2024

@arsaboo better to include the documents within the repository itself.
Reason being: if we mount local folders into a docker container, it can lead to permission issues & other complexity

@olafgeibig
Copy link

olafgeibig commented May 10, 2024

@ElishaKay @arsaboo I think the mounting a docker volume is a good idea. This is being used by many dockerized applications. Maybe it could be optional to keep the default simple. I guess very often production deployments wouldn't like to have their docs in git. Thanks for this work. Can't wait that it's merged.

@ElishaKay
Copy link
Contributor Author

ElishaKay commented May 10, 2024

2-minute internet and documents reports work after merging latest master.
5-minute reports need work :)

@ElishaKay ElishaKay closed this May 10, 2024
@ElishaKay ElishaKay reopened this May 10, 2024
@ElishaKay
Copy link
Contributor Author

ElishaKay commented May 12, 2024

@assafelovic,morning sir.

  • All 6 potential combos work: - 2-min, 5-min & resource-reports for both internet & document sources.
  • Don't believe it should have affected the multi-agent feature since I believe that folder was added after @proy9714's original branch verged off master.

And best of all.... zero conflicts with master 🎉
Would be thrilled if we could put other features on hold so as not to create new conflicts 😅

image

@proy9714 feel free to leave feedback here as well - solid stuff!
Let's do great things gentlemen

@assafelovic
Copy link
Owner

Thank you @proy9714 and @ElishaKay for this super super demanded feature! I'll stop all PRs until we get this through (except for hot fixes)! I will dive into it on Wed and we should release it next Monday

@assafelovic
Copy link
Owner

@ElishaKay can you update the documentation here: https://github.com/assafelovic/gpt-researcher/blob/master/docs/docs/gpt-researcher/tailored-research.md

build: ./
environment:
OPENAI_API_KEY: ${OPENAI_API_KEY}
TAVILY_API_KEY: ${TAVILY_API_KEY}
DOC_PATH: "=./docs/my-docs"
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@ElishaKay are you sure we should keep this static here? Probably keep it null as default?

Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

remove this directory

.gitignore Outdated
@@ -16,6 +16,9 @@ venv.bak/
#Ignore generated outputs
outputs/

#Ignore my local docs
docs/my-docs/
Copy link
Owner

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This shouldn't be in docs. Docs is intended for documentation files

Copy link
Owner

@assafelovic assafelovic left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is HUGE!

@assafelovic assafelovic merged commit 5c0f4e7 into assafelovic:master May 20, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants