Skip to content

Incomplete index for sparse files with non-parse parts > 8 GiB #156

@powellnorma

Description

@powellnorma

I created the .tar.zst with https://github.com/SaveTheRbtz/zstd-seekable-format-go - ratarmount loggs:

[Info] Try to open with rarfile
[Info] Try to open with tarfile
[Info] Detected compression zst for file object: <_io.BufferedReader name='archive.tar.zst'>
[Info] Undid zst file compression by using: IndexedZstdFile
Creating offset dictionary for archive.tar.zst ...
Position 902802605 of 112968843995 (0.80%). Remaining time: 8 min 8 s (current rate), 8 min 8 s (average rate). Spent time: 0 min 3 s
Position 1358774736 of 112968843995 (1.20%). Remaining time: 8 min 40 s (current rate), 8 min 17 s (average rate). Spent time: 0 min 6 s
Position 3794647863 of 112968843995 (3.36%). Remaining time: 7 min 50 s (current rate), 7 min 56 s (average rate). Spent time: 0 min 16 s
Position 5429518921 of 112968843995 (4.81%). Remaining time: 10 min 47 s (current rate), 8 min 42 s (average rate). Spent time: 0 min 26 s
Position 5916237504 of 112968843995 (5.24%). Remaining time: 7 min 19 s (current rate), 8 min 33 s (average rate). Spent time: 0 min 28 s
Position 6294844661 of 112968843995 (5.57%). Remaining time: 9 min 25 s (current rate), 8 min 35 s (average rate). Spent time: 0 min 30 s
Position 6720720693 of 112968843995 (5.95%). Remaining time: 8 min 32 s (current rate), 8 min 33 s (average rate). Spent time: 0 min 32 s
Position 7219378249 of 112968843995 (6.39%). Remaining time: 8 min 3 s (current rate), 8 min 28 s (average rate). Spent time: 0 min 34 s
Position 7689523853 of 112968843995 (6.81%). Remaining time: 7 min 42 s (current rate), 8 min 23 s (average rate). Spent time: 0 min 36 s
Position 7854005199 of 112968843995 (6.95%). Remaining time: 21 min 19 s (current rate), 8 min 39 s (average rate). Spent time: 0 min 38 s
Position 8305422074 of 112968843995 (7.35%). Remaining time: 7 min 44 s (current rate), 8 min 34 s (average rate). Spent time: 0 min 40 s
Position 9315929959 of 112968843995 (8.25%). Remaining time: 7 min 49 s (current rate), 8 min 24 s (average rate). Spent time: 0 min 45 s
Position 10249960142 of 112968843995 (9.07%). Remaining time: 7 min 42 s (current rate), 8 min 16 s (average rate). Spent time: 0 min 49 s
Position 10966962580 of 112968843995 (9.71%). Remaining time: 7 min 15 s (current rate), 8 min 9 s (average rate). Spent time: 0 min 52 s
Position 11644614155 of 112968843995 (10.31%). Remaining time: 7 min 34 s (current rate), 8 min 4 s (average rate). Spent time: 0 min 55 s
Position 12330155953 of 112968843995 (10.91%). Remaining time: 7 min 19 s (current rate), 7 min 58 s (average rate). Spent time: 0 min 58 s
Position 13223965556 of 112968843995 (11.71%). Remaining time: 7 min 25 s (current rate), 7 min 52 s (average rate). Spent time: 1 min 2 s
Position 15000695226 of 112968843995 (13.28%). Remaining time: 7 min 18 s (current rate), 7 min 41 s (average rate). Spent time: 1 min 10 s
Position 15229189612 of 112968843995 (13.48%). Remaining time: 14 min 15 s (current rate), 7 min 46 s (average rate). Spent time: 1 min 12 s
Resorting files by path ...
Creating offset dictionary for archive.tar.zst took 79.20s
[Info] The index does not yet contain zst block offset data. Will write it out.
Writing out TAR index to archive.tar.zst took 0s and is sized 31584256 B
[Info] Opened archive with tarfile backend.

It looks like it only indexes ~13%. When I use tar -I zstd -tvf archive.tar.zst all files get listed normally. Am I doing something wrong?

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions

      pFad - Phonifier reborn

      Pfad - The Proxy pFad of © 2024 Garber Painting. All rights reserved.

      Note: This service is not intended for secure transactions such as banking, social media, email, or purchasing. Use at your own risk. We assume no liability whatsoever for broken pages.


      Alternative Proxies:

      Alternative Proxy

      pFad Proxy

      pFad v3 Proxy

      pFad v4 Proxy