After update to 2.7.2 pathes missing and all documents #6404

hispeed88 · 2024-04-15T19:55:36Z

hispeed88
Apr 15, 2024

I updated to 2.7.2 and now I lost all documents in paperless but I have them in the folder. I don't see any paths so everything looks empty. For me it looks like I lost the database.

In the log i found this:

[2024-04-14 00:30:00,465] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/thumbnails/0000010.webp

[2024-04-14 00:30:00,466] [WARNING] [paperless.sanity_checker] Orphaned file in media dir: /usr/src/paperless/media/documents/originals/Basisordner/Diverses/Technik/Betriebsanleitung - Zyxel AX7501 Modem Router.pdf

[2024-04-14 01:05:00,128] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 02:05:00,150] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 03:05:00,124] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 04:05:00,126] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 05:05:00,153] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 06:05:00,127] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 07:05:00,126] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 08:05:02,032] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 09:05:00,122] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 10:05:00,133] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 11:05:00,131] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 12:05:00,127] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 13:05:00,128] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 14:05:00,147] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 15:05:00,126] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 16:05:00,125] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 17:05:00,124] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 18:05:00,125] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 19:05:00,125] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 20:05:00,132] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 21:05:00,129] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 22:05:00,127] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-14 23:05:00,127] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 00:05:00,120] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 01:05:00,133] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 02:05:00,133] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 03:05:00,127] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 04:05:00,124] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 05:05:00,127] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 06:05:00,128] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 07:05:00,123] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 08:05:00,128] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 09:05:00,122] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 10:05:00,130] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 11:05:00,125] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 12:05:00,125] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 13:05:00,127] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 14:05:00,129] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 15:05:00,136] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 16:05:00,130] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 17:05:00,129] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 18:05:00,187] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 19:05:00,130] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 20:05:00,130] [INFO] [paperless.tasks] No automatic matching items, not training

[2024-04-15 20:58:32,216] [DEBUG] [paperless.tasks] Skipping plugin CollatePlugin

[2024-04-15 20:58:32,217] [DEBUG] [paperless.tasks] Skipping plugin BarcodePlugin

[2024-04-15 20:58:32,217] [DEBUG] [paperless.tasks] Executing plugin WorkflowTriggerPlugin

[2024-04-15 20:58:32,223] [INFO] [paperless.tasks] WorkflowTriggerPlugin completed with:

[2024-04-15 20:58:32,286] [INFO] [paperless.consumer] Consuming Ticket - Eventfrog - Chäferfäscht 2024.pdf

[2024-04-15 20:58:32,306] [DEBUG] [paperless.consumer] Detected mime type: application/pdf

[2024-04-15 20:58:32,325] [DEBUG] [paperless.consumer] Parser: RasterisedDocumentParser

[2024-04-15 20:58:32,333] [DEBUG] [paperless.consumer] Parsing Ticket - Eventfrog - Chäferfäscht 2024.pdf...

[2024-04-15 20:58:32,375] [INFO] [paperless.parsing.tesseract] pdftotext exited 0

[2024-04-15 20:58:32,382] [DEBUG] [paperless.parsing.tesseract] Document has text, skipping OCRmyPDF entirely.

[2024-04-15 20:58:32,383] [DEBUG] [paperless.consumer] Generating thumbnail for Ticket - Eventfrog - Chäferfäscht 2024.pdf...

[2024-04-15 20:58:32,391] [DEBUG] [paperless.parsing] Execute: convert -density 300 -scale 500x5000> -alpha remove -strip -auto-orient -define pdf:use-cropbox=true /tmp/paperless/paperless-ngxhy9_psgb/Ticket - Eventfrog - Chäferfäscht 2024.pdf[0] /tmp/paperless/paperless-lgr305r2/convert.webp

[2024-04-15 20:58:34,981] [INFO] [paperless.parsing] convert exited 0

[2024-04-15 20:58:35,763] [DEBUG] [paperless.classifier] Document classification model does not exist (yet), not performing automatic matching.

[2024-04-15 20:58:35,771] [DEBUG] [paperless.consumer] Saving record to database

[2024-04-15 20:58:35,771] [DEBUG] [paperless.consumer] Creation date from parse_date: 2023-04-24 00:00:00+02:00

[2024-04-15 20:58:35,989] [DEBUG] [paperless.consumer] Deleting file /tmp/paperless/paperless-ngxhy9_psgb/Ticket - Eventfrog - Chäferfäscht 2024.pdf

[2024-04-15 20:58:36,002] [DEBUG] [paperless.parsing.tesseract] Deleting directory /tmp/paperless/paperless-lgr305r2

[2024-04-15 20:58:36,003] [INFO] [paperless.consumer] Document 2023-04-24 Ticket - Eventfrog - Chäferfäscht 2024 consumption finished

[2024-04-15 20:59:13,164] [DEBUG] [paperless.classifier] Document classification model does not exist (yet), not performing automatic matching.

[2024-04-15 21:00:20,978] [DEBUG] [paperless.classifier] Document classification model does not exist (yet), not performing automatic matching.

[2024-04-15 21:00:20,988] [ERROR] [paperless.api] System status detected a possible problem while loading the classifier: Classifier file does not exist (yet). Re-training may be pending.

Traceback (most recent call last):

  File "/usr/src/paperless/src/documents/views.py", line 1690, in get

    raise FileNotFoundError(classifier_error)

FileNotFoundError: Classifier file does not exist (yet). Re-training may be pending.

[2024-04-15 21:00:22,142] [DEBUG] [paperless.classifier] Document classification model does not exist (yet), not performing automatic matching.

[2024-04-15 21:00:22,152] [ERROR] [paperless.api] System status detected a possible problem while loading the classifier: Classifier file does not exist (yet). Re-training may be pending.

Traceback (most recent call last):

  File "/usr/src/paperless/src/documents/views.py", line 1690, in get

    raise FileNotFoundError(classifier_error)

FileNotFoundError: Classifier file does not exist (yet). Re-training may be pending.

[2024-04-15 21:00:23,269] [DEBUG] [paperless.classifier] Document classification model does not exist (yet), not performing automatic matching.

[2024-04-15 21:00:23,279] [ERROR] [paperless.api] System status detected a possible problem while loading the classifier: Classifier file does not exist (yet). Re-training may be pending.

Traceback (most recent call last):

  File "/usr/src/paperless/src/documents/views.py", line 1690, in get

    raise FileNotFoundError(classifier_error)

FileNotFoundError: Classifier file does not exist (yet). Re-training may be pending.

[2024-04-15 21:00:24,397] [DEBUG] [paperless.classifier] Document classification model does not exist (yet), not performing automatic matching.

[2024-04-15 21:00:24,408] [ERROR] [paperless.api] System status detected a possible problem while loading the classifier: Classifier file does not exist (yet). Re-training may be pending.

Traceback (most recent call last):

  File "/usr/src/paperless/src/documents/views.py", line 1690, in get

    raise FileNotFoundError(classifier_error)

FileNotFoundError: Classifier file does not exist (yet). Re-training may be pending.

[2024-04-15 21:01:35,337] [DEBUG] [paperless.classifier] Document classification model does not exist (yet), not performing automatic matching.

[2024-04-15 21:01:35,348] [ERROR] [paperless.api] System status detected a possible problem while loading the classifier: Classifier file does not exist (yet). Re-training may be pending.

Traceback (most recent call last):

  File "/usr/src/paperless/src/documents/views.py", line 1690, in get

    raise FileNotFoundError(classifier_error)

FileNotFoundError: Classifier file does not exist (yet). Re-training may be pending.

[2024-04-15 21:01:36,494] [DEBUG] [paperless.classifier] Document classification model does not exist (yet), not performing automatic matching.

[2024-04-15 21:01:36,505] [ERROR] [paperless.api] System status detected a possible problem while loading the classifier: Classifier file does not exist (yet). Re-training may be pending.

Traceback (most recent call last):

  File "/usr/src/paperless/src/documents/views.py", line 1690, in get

    raise FileNotFoundError(classifier_error)

FileNotFoundError: Classifier file does not exist (yet). Re-training may be pending.

[2024-04-15 21:01:37,629] [DEBUG] [paperless.classifier] Document classification model does not exist (yet), not performing automatic matching.

[2024-04-15 21:01:37,639] [ERROR] [paperless.api] System status detected a possible problem while loading the classifier: Classifier file does not exist (yet). Re-training may be pending.

Traceback (most recent call last):

  File "/usr/src/paperless/src/documents/views.py", line 1690, in get

    raise FileNotFoundError(classifier_error)

FileNotFoundError: Classifier file does not exist (yet). Re-training may be pending.

[2024-04-15 21:01:39,060] [DEBUG] [paperless.classifier] Document classification model does not exist (yet), not performing automatic matching.

[2024-04-15 21:01:39,071] [ERROR] [paperless.api] System status detected a possible problem while loading the classifier: Classifier file does not exist (yet). Re-training may be pending.

Traceback (most recent call last):

  File "/usr/src/paperless/src/documents/views.py", line 1690, in get

    raise FileNotFoundError(classifier_error)

FileNotFoundError: Classifier file does not exist (yet). Re-training may be pending.

[2024-04-15 21:01:54,213] [DEBUG] [paperless.classifier] Document classification model does not exist (yet), not performing automatic matching.

[2024-04-15 21:01:54,223] [ERROR] [paperless.api] System status detected a possible problem while loading the classifier: Classifier file does not exist (yet). Re-training may be pending.

Traceback (most recent call last):

  File "/usr/src/paperless/src/documents/views.py", line 1690, in get

    raise FileNotFoundError(classifier_error)

FileNotFoundError: Classifier file does not exist (yet). Re-training may be pending.

[2024-04-15 21:01:55,334] [DEBUG] [paperless.classifier] Document classification model does not exist (yet), not performing automatic matching.

[2024-04-15 21:01:55,343] [ERROR] [paperless.api] System status detected a possible problem while loading the classifier: Classifier file does not exist (yet). Re-training may be pending.

Traceback (most recent call last):

  File "/usr/src/paperless/src/documents/views.py", line 1690, in get

    raise FileNotFoundError(classifier_error)

FileNotFoundError: Classifier file does not exist (yet). Re-training may be pending.

[2024-04-15 21:04:18,527] [DEBUG] [paperless.management.consumer] Consumer exiting.

[2024-04-15 21:04:45,856] [INFO] [paperless.management.consumer] Polling directory for changes: /usr/src/paperless/consume

[2024-04-15 21:05:00,135] [DEBUG] [paperless.classifier] Document classification model does not exist (yet), not performing automatic matching.

[2024-04-15 21:05:00,146] [DEBUG] [paperless.classifier] Gathering data from database...

[2024-04-15 21:05:00,162] [DEBUG] [paperless.classifier] 1 documents, 0 tag(s), 0 correspondent(s), 0 document type(s). 0 storage path(es)

[2024-04-15 21:05:00,305] [DEBUG] [paperless.classifier] Vectorizing data...

[2024-04-15 21:05:00,819] [DEBUG] [paperless.classifier] There are no tags. Not training tags classifier.

[2024-04-15 21:05:00,819] [DEBUG] [paperless.classifier] There are no correspondents. Not training correspondent classifier.

[2024-04-15 21:05:00,820] [DEBUG] [paperless.classifier] There are no document types. Not training document type classifier.

[2024-04-15 21:05:00,820] [DEBUG] [paperless.classifier] There are no storage paths. Not training storage path classifier.

[2024-04-15 21:05:00,824] [INFO] [paperless.tasks] Saving updated classifier model to /usr/src/paperless/data/classification_model.pickle...

[2024-04-15 21:33:11,367] [DEBUG] [paperless.management.consumer] Consumer exiting.

[2024-04-15 21:33:36,277] [INFO] [paperless.management.consumer] Polling directory for changes: /usr/src/paperless/consume

[2024-04-15 21:37:29,215] [DEBUG] [paperless.management.consumer] Consumer exiting.

[2024-04-15 21:42:11,264] [INFO] [paperless.management.consumer] Polling directory for changes: /usr/src/paperless/consume

I have a docker enviroment. I repulled and redeployed the images with docker compose on portainer business edition.
How can I do some checks and what can I do now for the restore? Do need to create everything again?
What about the database i'm not sure if there everything works correctly at the moment.

This is the container for the database:

  db:
    container_name: paperless-db
    image: docker.io/library/postgres:16
    restart: always
    volumes:
      - /usr/src/paperless-ngx/paperless-db:/var/lib/containers/paperlessngx/postgresql/data
    environment:
      POSTGRES_DB: XXXXXX
      POSTGRES_USER: XXXXXX
      POSTGRES_PASSWORD: XXXXXXXXXXXXXXXXXXXXXX

Any ideas is apreciated.

Answered by shamoon

Apr 15, 2024

The code above suggests you mapped the wrong volume in your database container, the part on the right of the : is the internal path, so for Postgres it should be /var/lib/postgresql/data. Im not positive but I think if that container was stopped the data might be lost, if not perhaps it could still be recovered

View full answer

shamoon · 2024-04-15T20:02:06Z

shamoon
Apr 15, 2024
Maintainer

The code above suggests you mapped the wrong volume in your database container, the part on the right of the : is the internal path, so for Postgres it should be /var/lib/postgresql/data. Im not positive but I think if that container was stopped the data might be lost, if not perhaps it could still be recovered

0 replies

hispeed88 · 2024-04-16T18:53:56Z

hispeed88
Apr 16, 2024
Author

After a long working day I see that i was stupid and I really messed up this configuration.
Yes i restarted the container and since there is no database and I have no back up I lost it. The good thing is i made it correctly for the paperless/data
paperless/media
paperless/export
paperless/consume
folder but wrong for all other. I'm happy I have this problem now after 40 documents and not after 1000 documents.

0 replies

2024-05-17T03:03:48Z

github-actions[bot]
bot May 17, 2024

This discussion has been automatically locked since there has not been any recent activity after it was closed. Please open a new discussion for related concerns. See our contributing guidelines for more details.

0 replies

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

After update to 2.7.2 pathes missing and all documents #6404

{{title}}

{{editor}}'s edit

{{editor}}'s edit

Replies: 3 comments

{{title}}

{{title}}

{{title}}

Select a reply

After update to 2.7.2 pathes missing and all documents #6404

hispeed88 Apr 15, 2024

Replies: 3 comments

shamoon Apr 15, 2024 Maintainer

hispeed88 Apr 16, 2024 Author

github-actions[bot] bot May 17, 2024

hispeed88
Apr 15, 2024

shamoon
Apr 15, 2024
Maintainer

hispeed88
Apr 16, 2024
Author

github-actions[bot]
bot May 17, 2024