Mega refactor for optimized modularity 🎉🎆

[assafelovic]

We’re excited to release the next generation of GPT Researcher! We’ve completely refactored the code base to be more modular, customizable, stable and accurate. We’ve added many new features, improvements and bug fixes.

Below is a list of the main changes:

• Improved report generation prompt for better accuracy and quality.
• Added a new config structure including support for external JSON files.
• Redesigned GPT Researcher library to be enabled as a stand alone agent in any project (see example in repo).
• Added new structure for retrievers, enabling a better developer experience for adding and modifying information retrievers.
• Optimized configuration for latest GPT-4 Turbo model.
• Fixed issues with scraping and improved overall stability and speed.
• Added support for arxiv and pdf scraping urls.
• Added a rich documentation site with Docusaurus (see docs.tavily.com).
• Updated all Python packages to latest for most up to date performance and experience

Just git pull the latest version and give it a run!
Next, we’re building embedding support and long term memory!
To see what’s next on our roadmap check it out here: https://trello.com/b/3O7KBePw/gpt-researcher-roadmap

---

This discussion was created from the release Mega refactor for optimized modularity 🎉🎆.

[silopolis]

Looks like someone's been busy!
Just yesterday I was thinking how cool that could be to have GPT-Researcher integrated with Joplin, Dendron, Foam, or any other personal KB management system...
Hope that the "agent mode" is a big step to allow for this kind of integration.

Congratulations for the quality work
Keep on rocking

[assafelovic]

Thank you @silopolis super appreciate it! Can you please help us better understand on the use case you're aiming for? We'd love to consider it in the roadmap

[silopolis]

Hi,
As a (currently learning) DevOps engineer, I spend my life between:

• my web browser, with a huge (many Ks) bookmark built and cherished over the years, and a web clipping extension to capture content into my personnal KB
• my personnal KB, Joplin, for all my knowledge activities, from tidying up web clips, to writing notes and documentation documents of all sorts (many Ks)
• my IDE (Codium) but that's not relevant here

Having GPT-Researcher integrated with my PKB would allow me to benefit from its strength into my favorite tool each time I need to initiate a new research... Which can be very often depending the working phase I'm in.

Joplin already has a GPT plugin named Jarvis that works quite well. Told its dev about GPT-Researcher integration

[assafelovic]

That's awesome @silopolis thanks for sharing!