Gotenberg sometimes produces 400 when uploading docx files through "upload new documents" section

[egg82]

Description

Note that "sometimes" means that I'm dropping files directly from Google Drive, which I connected to my Rocky Linux 9 workstation via Gnome's "add Google account" feature. I'm sure there's other scenarios in which a failure occurs - likely with remotely-mounted systems.

The crux of the issue is that the files, when dropped onto the uploader, are renamed into strings like "1C3K-DJCcdf0gdHKlLiFkvHW89-IkWqSM" with no file extension. When src/paperless_tika/parsers.py's convert_to_pdf function is called, it attempts to parse out the file extension but fails since there is no extension. This causes a form upload error to Gotenberg.

Note that I have this behind an nginx proxy, but that shouldn't be the issue.

The exact error is:

Error while converting document to PDF: Client error '400 Bad Request' for url 'http://localhost:3000/forms/libreoffice/convert' For more information check: https://httpstatuses.com/400

Here's a log from Gotenberg:

docker logs gotenberg

  _____     __           __               
 / ___/__  / /____ ___  / /  ___ _______ _
/ (_ / _ \/ __/ -_) _ \/ _ \/ -_) __/ _ '/
\___/\___/\__/\__/_//_/_.__/\__/_/  \_, / 
                                   /___/

A Docker-powered stateless API for PDF files.
Version: 7.8.3
-------------------------------------------------------
[SYSTEM] modules: api chromium gc libreoffice logging pdfcpu pdfengines pdftk prometheus qpdf uno uno-pdfengine webhook 
[SYSTEM] uno: long-running LibreOffice listener ready to start
[SYSTEM] gc: application started
[SYSTEM] prometheus: collecting metrics
[SYSTEM] pdfengines: pdfcpu pdftk qpdf uno-pdfengine
[SYSTEM] api: server listening on port 3000
{"level":"error","ts":1700361646.2566292,"logger":"api","msg":"validate form data: no form file found for extensions: [.bib .doc .xml .docx .fodt .html .ltx .txt .odt .ott .pdb .pdf .psw .rtf .sdw .stw .sxw .uot .vor .wps .epub .png .bmp .emf .eps .fodg .gif .jpg .jpeg .met .odd .otg .pbm .pct .pgm .ppm .ras .std .svg .svm .swf .sxd .sxw .tif .tiff .xhtml .xpm .odp .fodp .potm .pot .pptx .pps .ppt .pwp .sda .sdd .sti .sxi .uop .wmf .csv .dbf .dif .fods .ods .ots .pxl .sdc .slk .stc .sxc .uos .xls .xlt .xlsx .odg .dotx .xltx]","trace":"9ec46388-f494-4625-b2fe-cdf72340b292","remote_ip":"172.17.0.1","host":"localhost:3000","uri":"/forms/libreoffice/convert","method":"POST","path":"/forms/libreoffice/convert","referer":"","user_agent":"python-httpx/0.24.1","status":400,"latency":3092993,"latency_human":"3.092993ms","bytes_in":224386,"bytes_out":466}

Config alterations are as follows:

PAPERLESS_REDIS=redis://localhost:6379
PAPERLESS_DBENGINE=mariadb
PAPERLESS_DBHOST=localhost
PAPERLESS_DBPORT=3306
PAPERLESS_DBNAME={{ paperless_db_name }}
PAPERLESS_DBUSER={{ paperless_db_user }}
PAPERLESS_DBPASS={{ paperless_db_pass }}
PAPERLESS_DBSSLMODE=PREFERRED

PAPERLESS_CONSUMPTION_DIR=/mnt/nas/paperless/consume
PAPERLESS_MEDIA_ROOT=/mnt/nas/paperless/media

PAPERLESS_SECRET_KEY={{ lookup('password', '/dev/null chars=ascii_letters,digits length=50') }}
PAPERLESS_URL=https://{{ ansible_facts['fqdn'] }}
PAPERLESS_CSRF_TRUSTED_ORIGINS=https://{{ ansible_facts['fqdn'] }} # can be set using PAPERLESS_URL
PAPERLESS_ALLOWED_HOSTS={{ ansible_facts['fqdn'] }} # can be set using PAPERLESS_URL
PAPERLESS_CORS_ALLOWED_HOSTS=https://localhost:8080,https://{{ ansible_facts['fqdn'] }} # can be set using PAPERLESS_URL

PAPERLESS_ADMIN_USER={{ paperless_admin_user }}
PAPERLESS_ADMIN_PASSWORD={{ paperless_admin_pass }}
PAPERLESS_USE_X_FORWARD_HOST=true
USE_X_FORWARD_PORT=true

PAPERLESS_TASK_WORKERS=2
PAPERLESS_THREADS_PER_WORKER=4
PAPERLESS_ENABLE_COMPRESSION=0
PAPERLESS_CONSUMER_RECURSIVE=true
PAPERLESS_CONSUMER_ENABLE_BARCODES=true
PAPERLESS_CONSUMER_ENABLE_ASN_BARCODE=true
PAPERLESS_CONSUMER_BARCODE_UPSCALE=1.5
PAPERLESS_CONSUMER_BARCODE_DPI=600
PAPERLESS_CONSUMER_COLLATE_DOUBLE_SIDED_TIFF_SUPPORT=true
PAPERLESS_THUMBNAIL_FONT_NAME=/usr/share/fonts/liberation-serif/LiberationSerif-Regular.ttf
PAPERLESS_CONSUMER_POLLING=60
PAPERLESS_ENABLE_NLTK=1
PAPERLESS_NLTK_DIR=/usr/share/nltk_data

PAPERLESS_TIKA_ENABLED=true
PAPERLESS_TIKA_ENDPOINT=http://localhost:9998
PAPERLESS_TIKA_GOTENBERG_ENDPOINT=http://localhost:3000

and the logs from the webserver are below.

Steps to reproduce

1. Create a Rocky 9 workstation
2. Attach GDrive via Gnome's "add account" feature
3. Go to dashboard
4. Upload docx file by dragging and dropping one onto the "Upload new documents" section directly from GDrive (not from a local filesystem)
5. See failure

Webserver logs

[2023-11-19 02:40:46,087] [INFO] [paperless.parsing.tika] Sending /tmp/paperless/paperless-ngx1ubnc6k_/1C3K-DJCcdf0gdHKlLiFkvHW89-IkWqSM to Tika server

[2023-11-19 02:40:46,212] [INFO] [paperless.parsing.tika] Converting /tmp/paperless/paperless-ngx1ubnc6k_/1C3K-DJCcdf0gdHKlLiFkvHW89-IkWqSM to PDF as /tmp/paperless/paperless-kymv6rog/convert.pdf

[2023-11-19 02:40:46,261] [ERROR] [paperless.consumer] Error occurred while consuming document 1C3K-DJCcdf0gdHKlLiFkvHW89-IkWqSM: Error while converting document to PDF: Client error '400 Bad Request' for url 'http://localhost:3000/forms/libreoffice/convert'

For more information check: https://httpstatuses.com/400

Traceback (most recent call last):

  File "/opt/paperless/src/paperless_tika/parsers.py", line 105, in convert_to_pdf

    response.raise_for_status()  # ensure we notice bad responses

  File "/usr/local/lib/python3.9/site-packages/httpx/_models.py", line 749, in raise_for_status

    raise HTTPStatusError(message, request=request, response=self)

httpx.HTTPStatusError: Client error '400 Bad Request' for url 'http://localhost:3000/forms/libreoffice/convert'

For more information check: https://httpstatuses.com/400

The above exception was the direct cause of the following exception:

Traceback (most recent call last):

  File "/opt/paperless/src/documents/consumer.py", line 425, in try_consume_file

    document_parser.parse(self.path, mime_type, self.filename)

  File "/opt/paperless/src/paperless_tika/parsers.py", line 70, in parse

    self.archive_path = self.convert_to_pdf(document_path, file_name)

  File "/opt/paperless/src/paperless_tika/parsers.py", line 107, in convert_to_pdf

    raise ParseError(

documents.parsers.ParseError: Error while converting document to PDF: Client error '400 Bad Request' for url 'http://localhost:3000/forms/libreoffice/convert'

For more information check: https://httpstatuses.com/400

Browser logs

No response

Paperless-ngx version

1.17.4

Host OS

Rocky Linux release 9.2 5.14.0-284.30.1.el9_2.x86_64

Installation method

Bare metal

Browser

Firefox 115.4.0esr

Configuration changes

No response

Other

No response

[github-actions[bot]]

This discussion has been automatically closed due to inactivity. Please see our contributing guidelines for more details.