Support storing UnixFS 1.5 Mode and ModTime #7754

kstuart · 2020-11-05T13:47:38Z

Feature Progress

Related PRs

Implementation Notes

When adding files and directories without opting to store a mode or modification time the same CIDs are generated that would have been created before this feature was implemented (opt-in).
The Go runtime currently has no native support for restoring file mode and modification time on symbolic-links, support for restoring the last modification time has been added for Linux distributions and the following BSDs: freebsd, netbsd, openbsd, dragonflybsd.
Automatically updating a modification time will only occur if a modification time was previously stored.
When creating an archive, for compatibility, time resolution is to the second; Nanoseconds are not supported.

IPFS Add (CLI)

The ipfs add options --preserve-mode and --preserve-mtime are used to store the original mode and last modified time of the file being added, the options --mode, --mtime and --mtime-nsecs are used to store custom values, a custom value of 0 is a no-op as is providing --mtime-nsecs without --mtime.

The preserve flags and custom options are mutually exclusive, if both are provided the custom options take precedence.

Closes #6920

This commit introduces initial Mode and ModTime support for single filesystem files and webfiles. The ipfs add options --preserve-mode and --preserve-mtime are used to store the original mode and last modified time of the file being added, the options --mode, --mtime and --mtime-nsecs are used to store custom values. A custom value of 0 is a no-op. The preserve flags and custom options are mutually exclusive, if both are provided the custom options take precedence.

welcome · 2020-11-05T13:47:40Z

Thank you for submitting this PR!
A maintainer will be here shortly to review it.
We are super grateful, but we are also overloaded! Help us by making sure that:

The context for this PR is clear, with relevant discussion, decisions
and stakeholders linked/mentioned.
Your contribution itself is clear (code comments, self-review for the
rest) and in its best form. Follow the code contribution
guidelines
if they apply.

Getting other community members to do a review would be great help too on complex PRs (you can ask in the chats/forums). If you are unsure about something, just leave us a comment.
Next steps:

A maintainer will triage and assign priority to this PR, commenting on
any missing things and potentially assigning a reviewer for high
priority items.
The PR gets reviews, discussed and approvals as needed.
The PR is merged by maintainers when it has been approved and comments addressed.

We currently aim to provide initial feedback/triaging within two business days. Please keep an eye on any labelling actions, as these will indicate priorities and status of your contribution.
We are very grateful for your contribution!

This is needed for when ipfs get is used with the --output option to ensure the target for applying metadata is known.

…n time

…ixfs/ufs15 � Conflicts: � core/commands/files.go

lidel

While having feature parity with JS and also UnixFS Specs (draft in ipfs/specs#331) would be nice, this was always a low priority, and slipped during triages. Let's see if we could land this.

@hacdias will look into this and ipfs/boxo#34 and see if we can rebase and ship this in 0.30 without investing too much time.

Some comments inline.

lidel · 2024-05-17T15:07:32Z

core/commands/add.go

+	preserveModeOptionName  = "preserve-mode"
+	preserveMtimeOptionName = "preserve-mtime"
+	modeOptionName          = "mode"
+	mtimeOptionName         = "mtime"
+	mtimeNsecsOptionName    = "mtime-nsecs"


💭 Is it possible to preserve mtime without preserving mode?

To avoid cross-implementation issues, we should have "zero values" documented in https://github.com/ipfs/specs/pull/331/files

lidel · 2024-05-17T15:28:30Z

core/commands/add.go

+		cmds.BoolOption(preserveModeOptionName, "Apply permissions to created UnixFS entries"),
+		cmds.BoolOption(preserveMtimeOptionName, "Apply modification time to created UnixFS entries"),
+		cmds.UintOption(modeOptionName, "File mode to apply to created UnixFS entries"),
+		cmds.Int64Option(mtimeOptionName, "Modification time in seconds before or after the Unix Epoch to apply to created UnixFS entries"),
+		cmds.UintOption(mtimeNsecsOptionName, "Modification time fraction in nanoseconds"),


Suggested change

cmds.BoolOption(preserveModeOptionName, "Apply permissions to created UnixFS entries"),

cmds.BoolOption(preserveMtimeOptionName, "Apply modification time to created UnixFS entries"),

cmds.UintOption(modeOptionName, "File mode to apply to created UnixFS entries"),

cmds.Int64Option(mtimeOptionName, "Modification time in seconds before or after the Unix Epoch to apply to created UnixFS entries"),

cmds.UintOption(mtimeNsecsOptionName, "Modification time fraction in nanoseconds"),

cmds.BoolOption(preserveModeOptionName, "Apply existing POSIX permissions to created UnixFS entries"),

cmds.BoolOption(preserveMtimeOptionName, "Apply existing POSIX modification time to created UnixFS entries"),

cmds.UintOption(modeOptionName, "Custom POSIX file mode to store in created UnixFS entries"),

cmds.Int64Option(mtimeOptionName, "Custom POSIX modification time to store in created UnixFS entries (seconds before or after the Unix Epoch)"),

cmds.UintOption(mtimeNsecsOptionName, "Custom POSIX modification time (optional time fraction in nanoseconds)"),

lidel · 2024-05-17T15:30:20Z

core/commands/files.go

+		Tagline: "Change mode permissions",
+		ShortDescription: `
+The mode argument must be specified in unix numeric notation. 


Suggested change

Tagline: "Change mode permissions",

ShortDescription: `

The mode argument must be specified in unix numeric notation.

Tagline: "Change optional POSIX mode permissions",

ShortDescription: `

The mode argument must be specified in Unix numeric notation.

lidel · 2024-05-17T15:31:04Z

core/commands/files.go

+
+var filesTouchCmd = &cmds.Command{
+	Helptext: cmds.HelpText{
+		Tagline: "Change file modification times.",


Suggested change

Tagline: "Change file modification times.",

Tagline: "Change optional POSIX modification times.",

lidel · 2024-05-17T15:38:23Z

core/commands/files.go

 }

 const (
 	defaultStatFormat = `<hash>
 Size: <size>
 CumulativeSize: <cumulsize>
 ChildBlocks: <childs>
-Type: <type>`
+Type: <type>
+Mode: <mode>


💭 seems <mode-octal> would only be used with --format.

Perhaps it will be useful to show it here as well?

Suggested change

Mode: <mode>

Mode: <mode> (<mode-octal>)

@kstuart

Replaces #7754 written by @kstuart - ipfs/boxo#653 - ipfs/boxo#658 - [X] Can `ipfs add` with preserved mode and/or last modification time - [X] on files - [X] on directories - [X] Can `ipfs add` with custom mode and/or last modification time - [X] on files - [X] on directories - [X] Can `ipfs get` restoring mode and/or last modification time - [X] on files - [X] on directories - [X] in archives - [X] Can `ipfs files chmod` to change mode - [X] on files - [X] on directories - [X] Can `ipfs files touch` to change last modification time - [X] on files - [X] on directories - [X] Automatically update the last modification time when file data is changed or truncated (e.g. `ipfs files write`) - [X] Can add files and directories with mode and/or modification time using multipart-form data - [X] `ipfs files stat` reports mode and last modification time **Note:** - [X] Adds support to `kubo/core/rpc` (may require additional tests). - ~ipfs/interface-go-ipfs-core/pull/66~ replace by this PR - ~ipfs/go-unixfs/pull/85~ replaced by: ipfs/boxo#658 - ~ipfs/go-mfs/pull/93~ replaced by: ipfs/boxo#658 - ~ipfs/go-ipfs-files/pull/31~ replaced by: ipfs/boxo#653 - ~ipfs/tar-utils/pull/11~ replaced by: ipfs/boxo#653 - When adding files and directories without opting to store a mode or modification time the same CIDs are generated that would have been created before this feature was implemented (opt-in). - The Go runtime currently has no native support for restoring file mode and modification time on symbolic-links, support for restoring the last modification time has been added for Linux distributions and the following BSDs: freebsd, netbsd, openbsd, dragonflybsd. - Automatically updating a modification time will only occur if a modification time was previously stored. - When creating an archive, for compatibility, time resolution is to the second; Nanoseconds are not supported. The `ipfs add` options `--preserve-mode` and `--preserve-mtime` are used to store the original mode and last modified time of the file being added, the options `--mode`, `--mtime` and `--mtime-nsecs` are used to store custom values, a custom value of 0 is a no-op as is providing `--mtime-nsecs` without `--mtime`. The preserve flags and custom options are mutually exclusive, if both are provided the custom options take precedence. --- Closes #6920

This commit introduces initial Mode and ModTime support for single filesystem files and webfiles. The ipfs add options --preserve-mode and --preserve-mtime are used to store the original mode and last modified time of the file being added, the options --mode, --mtime and --mtime-nsecs are used to store custom values. A custom value of 0 is a no-op. The preserve flags and custom options are mutually exclusive, if both are provided the custom options take precedence. Majority of of the code was from #7754 written by kstuart Replaces #7753 Closes #6920

lidel · 2024-08-13T15:39:29Z

Continued in #10478

kstuart mentioned this pull request Nov 5, 2020

[BOUNTY] Implement UnixFSv1.5 in go-ipfs #6920

Open

kstuart added 11 commits November 10, 2020 19:33

[ipfs get] pass the output path to FileArchive

d93f1ef

This is needed for when ipfs get is used with the --output option to ensure the target for applying metadata is known.

[sharness] add tests for preserving/setting file mode and modificatio…

0d883e9

…n time

Merge branch 'master' into feat/unixfs/ufs15

a85c903

add file mode and mtime support for file stat

9d257fe

[ipfs add] emit mtime and mode in AddEvent

659e709

add optional mode and mtime to ipfs files stat output

e1d9879

Merge branch 'master' of https://github.com/ipfs/go-ipfs into feat/un…

ddf9073

…ixfs/ufs15 � Conflicts: � core/commands/files.go

humanize mode and mtime in ipfs files stat output

16b57a6

don't set modification time when none provided

45713c2

fix/enhance files stat template

9172385

add ipfs files chmod and touch

a6f5f09

kstuart mentioned this pull request Sep 6, 2021

initial support for setting/updating file modes and modification times ipfs/go-mfs#93

Closed

kstuart added 6 commits September 9, 2021 22:02

make files stat json output compatible with js-ipfs again

d7e61f0

Merge remote-tracking branch 'upstream/master' into feat/unixfs/ufs15

ae2d3d4

fix bad format of JSON statOutput.Mode

46e0660

remove leftover code

29860a1

preserve mode/mtime on directories and symlinks

e40cf41

correct comment

fd80be0

kstuart changed the title ~~preliminary support for unixfs Mode and ModTime~~ Support storing UnixFS 1.5 Mode and ModTime Jun 26, 2022

kstuart added 7 commits June 26, 2022 20:10

remove unneeded parameter

ae46f6a

remove unneeded cast

fb8d100

sync with upstream master brach

574b656

finalize files stat format for mode and mtime

c000653

fix storing mode and mtime on symlinks and directories

03a37d6

complete sharness tests for mode and modification time

ed2b842

add missing test and minor fixes

8b3f27d

kstuart marked this pull request as ready for review October 31, 2022 14:07

lidel reviewed May 17, 2024

View reviewed changes

lidel assigned hacdias May 17, 2024

lidel requested a review from hacdias May 17, 2024 15:42

lidel mentioned this pull request May 17, 2024

feat: support UnixFS 1.5 file mode and modification times ipfs/boxo#34

Closed

5 tasks

hacdias removed their assignment May 27, 2024

hacdias removed their request for review May 27, 2024 12:41

lidel mentioned this pull request May 28, 2024

Release 0.30 #10436

Open

29 tasks

gammazero mentioned this pull request Aug 13, 2024

feat: Support storing UnixFS 1.5 Mode and ModTime #10478

Open

23 tasks

lidel closed this Aug 13, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Support storing UnixFS 1.5 Mode and ModTime #7754

Support storing UnixFS 1.5 Mode and ModTime #7754

	Tagline: "Change file modification times.",
	Tagline: "Change optional POSIX modification times.",

Support storing UnixFS 1.5 Mode and ModTime #7754

Support storing UnixFS 1.5 Mode and ModTime #7754

Conversation

Feature Progress

Related PRs

Implementation Notes

IPFS Add (CLI)

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment