The in-docs links (e.g., links to types such as “string”, “attribute set”, etc.) are just empty ones for now.
The inspiration for the “pseudo types” section come from the Erlang docs that is also a dynamically typed language, but has a “sub-language” for type specification for functions and each function doc starts with these, thus making the descriptions unambiguous.
improvement ideas:
Make “Types” section collapsible and add hover tooltip to expand type mentions in the descriptive portion of this reference.
Fix table 2.3-1’s vertical header to the left so that it doesn’t scroll out of screen
Link footnotes
Make typespec notes less prominent (italics doesn’t seem to do much. CSS?)
fileURL :: string = "file://" + fileURLPathPart fileURLPathPart = string fileURLPathPart is a shorthand for fileURL (i.e., it will be prefixed with "file://" during evaluation) therefore both need to conform to the file:// URI scheme (see the path syntax of RFC 8089).
gitArgs attributes rev and ref will only be discussed in subsequent sections, but they also needed to be addressed here because of the significant role they play regarding the call results.
Calls with pathLike arguments attempt to fetch a repo in a directory on a local or remote file system. The target repo may be a project under active development so their status and state may need to be determined before trying to copy the repo to the Nix store.
1.2.1 Git repository characteristics
That is, characteristics that builtins.fetchGit cares about.
1.2.1.1 Status
The status of a Git repo is
dirty, if there are modified tracked files and/or staged changes.
(Untracked content does not count.)
clean, if the output of git diff-index HEAD is empty. (If there are only untracked files in git status, the repo is clean.)
1.2.1.2 State
The state of a Git repo is the specific commit where the HEAD reference points to (directly or indirectly) at the moment when the repo is fetched.
Directly, if the repo is in a “detached HEAD” state, and indirectly when the commit is also the target of other references as shown on the figure below.
1.2.1.2-1. State of a Git repo
LEGEND: orange label = branch, blue label = tag
1.2.2. Argument of type Nix path, fileURL, or fileURLPathPart
</tbody>
</table>
In fact, the 3 "STATE" rows could easily be collapsed into one as Git branches and tags are only labels to a Git object and what matters to `fetchGit` is the specific commit at the end of the de-reference process.
Example calls:
+ via [Nix path]():\
`builtins.fetchGit ~/clones/nix`
+ via **fileURL**:\
`builtins.fetchGit "file:///home/nix_user/clones/nix"`
+ via **fileURLPathPart**:\
`builtins.fetchGit "/home/nix_user/clones/nix"`
#### 1.2.3 `pathLikeGitArgs` type argument
This means one of the following:
+ via { `url` :: [Nix path]() }
`builtins.fetchGit { url = ~/clones/nix; ... }`
+ via { `url` :: **fileURLPathPart** }
`builtins.fetchGit { url = "/home/nix_user/clones/nix"; ... }`
+ via { `url` :: **fileURL** }
`builtins.fetchGit { url = "file:///home/nix_user/clones/nix"; ... }`
The following table takes advantage of the fact that [state]() is simply determined by the current value of the [HEAD reference][HEAD]:
> NOTE
>
> [`gitArgs`]() attributes [`rev`]() and [`ref`]() will only be discussed in subsequent sections, but they also needed to be addressed here because of the significant role they play regarding the call results.
</table>
\[1.2.3-1]: See section [3.3 `rev`]()
\[1.2.3-2]: When `ref` or `rev` is present, the intention is probably to fetch a known past state from the repo's history, thus the most recent changes are not relevant (neither the status of the repo).\[1.2.3-3]: See section [3.4 `ref`]()
As a corollary, here are some tips:
+ If you need to fetch a local repo, calling `builtins.fetchGit` with `ref` (branch or tag) or `rev` (commit hash) will make sure that a repo is fetched with a predictable content, ignoring any changes that may have been made since you last touched it.
+ If you are packaging a project under active development and want to test changes without commiting, you'll probably want to call `builtins.fetchGit` with `{ url = ...; }` or the specified in [1.2.2. Argument of type `Nix path`, `fileURL`, or `fileURLPathPart`]().
## 2. **gitArgs** attributes
Reminder:
**gitArgs** :: [attribute set]() =\
{ `url` :: (**URL** | **path**);\
[ `name` :: [string]() ? `"source"` ];\
[ `ref` :: **gitReference** ? `"HEAD"` ];\
[ `rev` :: **gitFullCommitHash** ? <`ref` dereferenced> ];\
[ `submodules` :: [boolean]() ? `false` ];\
[ `shallow` :: [boolean]() ? `false` ];\
[ `allRefs` :: [boolean]() ? `false` ];\
}
### 3.1 `url` (mandatory)
The rev attribute is used to refer to a specific commit by the full SHA-1 Git object name (40-byte hexadecimal string) - or as it is more colloquially called, the commit hash.
The dereferenced value of the Git reference held by the ref attribute. (See next section.)
</table>
Sections [1.1.2 **webLikeGitArgs** type argument]() and [1.2.3 **pathLikeGitArgs** type argument]()) in [1. Behaviour]() describe the prevailing behaviour `builtins.fetchgit` when the `rev` attribute is used.
> NOTE
>
> Specifying the `rev` attribute will render the `ref` attribute irrelevant no matter if it is included in the input attribute set or not. See next section for more.
### 3.4 `ref` (optional)
Description
The ref attribute accepts a Git reference that is present in the target repo.
</table>
> WARNING
>
> By default, the `ref` value is prefixed with `refs/heads/`. After Nix 2.3.0, it will not be prefixed with `refs/heads/` if `ref` starts with `refs/`.
#### 3.3.1 [`ref` attribute]() ignored when the `rev` attribute is provided
The `rev` attribute (i.e., the commit hash) has higher specificity; a `ref` reference will need to be resolved and its value may change with time, but a commit hash will always point to the same exact commit object and thus to the same state of the the repo during the lifetime of a Git repo. (TODO: right?)
## 4. Examples
> TODO: Re-work original examples
---
TODO/NOTE: Stopping here for now to wait for the resolution of [comment on Nix issue #5128](https://github.com/NixOS/nix/issues/5128#issuecomment-1198254451)
Here are some examples of how to use `builtins.fetchGit`.
- To fetch a private repository over SSH:
```nix
builtins.fetchGit {
url = "git@github.com:my-secret/repository.git";
ref = "master";
rev = "adab8b916a45068c044658c4158d81878f9ed1c3";
}
```
- To fetch an arbitrary reference:
```nix
builtins.fetchGit {
url = "https://github.com/NixOS/nix.git";
ref = "refs/heads/0.5-release";
}
```
- If the revision you're looking for is in the default branch of
the git repository you don't strictly need to specify the branch
name in the `ref` attribute.
However, if the revision you're looking for is in a future
branch for the non-default branch you will need to specify the
the `ref` attribute as well.
```nix
builtins.fetchGit {
url = "https://github.com/nixos/nix.git";
rev = "841fcbd04755c7a2865c51c1e2d3b045976b7452";
ref = "1.11-maintenance";
}
```
> **Note**
>
> It is nice to always specify the branch which a revision
> belongs to. Without the branch being specified, the fetcher
> might fail if the default branch changes. Additionally, it can
> be confusing to try a commit from a non-default branch and see
> the fetch fail. If the branch is specified the fault is much
> more obvious.
- If the revision you're looking for is in the default branch of
the git repository you may omit the `ref` attribute.
```nix
builtins.fetchGit {
url = "https://github.com/nixos/nix.git";
rev = "841fcbd04755c7a2865c51c1e2d3b045976b7452";
}
```
- To fetch a specific tag:
```nix
builtins.fetchGit {
url = "https://github.com/nixos/nix.git";
ref = "refs/tags/1.9";
}
```
- To fetch the latest version of a remote branch:
```nix
builtins.fetchGit {
url = "ssh://git@github.com/nixos/nix.git";
ref = "master";
}
```
> **Note**
>
> Nix will refetch the branch in accordance with
> the option `tarball-ttl`.
> **Note**
>
> This behavior is disabled in *Pure evaluation mode*.
TODO: move this to the end
\[2.2-1]: See [`src/libfetchers/git.cc`#`445`](https://github.com/NixOS/nix/blob/86fcd4f6923b3a8ccca261596b9db0d8c0a873ec/src/libfetchers/git.cc#L445-L452).
\[2.2-2]: See [`src/libfetchers/git.cc`#`fetch()`](https://github.com/NixOS/nix/blob/86fcd4f6923b3a8ccca261596b9db0d8c0a873ec/src/libfetchers/git.cc#L393), line [556](https://github.com/NixOS/nix/blob/86fcd4f6923b3a8ccca261596b9db0d8c0a873ec/src/libfetchers/git.cc#L556).
+ `submodules` (optional)
+ `shallow` (optional)
+ `allRef` (optional)
- `ref` (optional)\
The git ref to look for the requested revision under. This is
often a branch or tag name. Defaults to `HEAD`.
- `rev` (optional)\
The [Git revision](https://git-scm.com/docs/git-rev-parse#_specifying_revisions) to fetch.\
*Default value*: if the `ref` attribute (see above) is specified,
- `submodules` (optional)\
A Boolean parameter that specifies whether submodules should be
checked out. Defaults to `false`.
- `shallow` (optional)\
A Boolean parameter that specifies whether fetching a shallow clone
is allowed. Defaults to `false`.
NOTE [`git clone --depth=1 ` creates a shallow clone](https://github.blog/2020-12-21-get-up-to-speed-with-partial-clone-and-shallow-clone/#:~:text=git%20clone%20%2D%2Ddepth%3D1%20%3Curl%3E%C2%A0creates%20a%C2%A0shallow%20clone)
QUESTION How does this affect rev and ref? this is what i think
+ "no ref, no rev": ref=HEAD rev=resolve(HEAD)
+++++++++clean repo: only diff is revCount
The store hash stayed the same! Probably because .git is never copied - but then what is the point of this switch because then it will always be shallow.
https://stackoverflow.com/questions/11497457/git-clone-without-git-directory
QUESTION also, if .git is never copied, what is the point of revcount?
NOTE should be called commitCount
nix-repl> builtins.fetchGit { url = ~/shed/my-project; }
{ lastModified = 1658846311; lastModifiedDate = "20220726143831"; narHash = "sha256-Yph6eCPxkG4TeoDAh/W6xaG+j5oFAui80c1FMYaGPTY="; outPath = "/nix/store/waffpfm7xrfyh1yj60v4phaf49ccyjd0-source"; rev = "5f45e9c854941c72deb9d36fb3e95e4feb4d698f"; revCount = 5; shortRev = "5f45e9c"; submodules = false; }
nix-repl> builtins.fetchGit { url = ~/shed/my-project; shallow = true; }
{ lastModified = 1658846311; lastModifiedDate = "20220726143831"; narHash = "sha256-Yph6eCPxkG4TeoDAh/W6xaG+j5oFAui80c1FMYaGPTY="; outPath = "/nix/store/waffpfm7xrfyh1yj60v4phaf49ccyjd0-source"; rev = "5f45e9c854941c72deb9d36fb3e95e4feb4d698f"; revCount = 0; shortRev = "5f45e9c"; submodules = false; }
---------
+++++++++dirty repo: completely identical outputs
nix-repl> builtins.fetchGit { url = ./.; }
warning: Git tree '/home/toraritte/clones/nix' is dirty
{ lastModified = 1658888196; lastModifiedDate = "20220727021636"; narHash = "sha256-bcPz3nYp1zK0HwwBqEybsxpDs5V7TPGUP3RE3Myd8zM="; outPath = "/nix/store/g0mrkf6vq0w6qzbkj03f4z3qhx18w50n-source"; rev = "0000000000000000000000000000000000000000"; revCount = 0; shortRev = "0000000"; submodules = false; }
nix-repl> builtins.fetchGit { url = ./.; shallow = true; }
warning: Git tree '/home/toraritte/clones/nix' is dirty
{ lastModified = 1658888196; lastModifiedDate = "20220727021636"; narHash = "sha256-bcPz3nYp1zK0HwwBqEybsxpDs5V7TPGUP3RE3Myd8zM="; outPath = "/nix/store/g0mrkf6vq0w6qzbkj03f4z3qhx18w50n-source"; rev = "0000000000000000000000000000000000000000"; revCount = 0; shortRev = "0000000"; submodules = false; }
+++++++++stopping this experiment here because adding rev and/or ref doesn't make a difference: only revCount will differ
- `allRefs` (optional)\
Whether to fetch all refs of the repository. With this argument being
true, it's possible to load a `rev` from *any* `ref` (by default only
`rev`s from the specified `ref` are supported).
NOTE allRefs also seems kind of pointless
nix-repl> builtins.fetchGit { url = ~/shed/my-project; allRefs = true; ref = "main";}
{ lastModified = 1658846059; lastModifiedDate = "20220726143419"; narHash = "sha256-FQUE8ek9uoyMy
uGjQirYVc5B16X1Uq/k5e4LH+yv4S4="; outPath = "/nix/store/3ra7y3vsnbz707nq8r2d5p7k4irmiwrp-source";
rev = "c277976fce0b2b32b954a66d4345730b5b08f1db"; revCount = 3; shortRev = "c277976"; submodules
= false; }
nix-repl> builtins.fetchGit { url = ~/shed/my-project; ref = "main";}
{ lastModified = 1658846059; lastModifiedDate = "20220726143419"; narHash = "sha256-FQUE8ek9uoyMy
uGjQirYVc5B16X1Uq/k5e4LH+yv4S4="; outPath = "/nix/store/3ra7y3vsnbz707nq8r2d5p7k4irmiwrp-source";
rev = "c277976fce0b2b32b954a66d4345730b5b08f1db"; revCount = 3; shortRev = "c277976"; submodules
= false; }
nix-repl> builtins.fetchGit { url = "https://github.com/nixos/nix"; }
{ lastModified = 1658489272; lastModifiedDate = "20220722112752"; narHash = "sha256-z0ov/NPT8egao
DUVw4i5SuKcx6t7YZbL7lzdOBsP1sA="; outPath = "/nix/store/z13wfalqlfshjbkx5kwwgfm3350xnpdx-source";
rev = "280543933507839201547f831280faac614d0514"; revCount = 12454; shortRev = "2805439"; submod
ules = false; }
nix-repl> builtins.fetchGit { url = "https://github.com/nixos/nix"; allRef = true; }
error: unsupported Git input attribute 'allRef'
nix-repl> builtins.fetchGit { url = "https://github.com/nixos/nix"; allRefs = true; }
{ lastModified = 1658489272; lastModifiedDate = "20220722112752"; narHash = "sha256-z0ov/NPT8egao
DUVw4i5SuKcx6t7YZbL7lzdOBsP1sA="; outPath = "/nix/store/z13wfalqlfshjbkx5kwwgfm3350xnpdx-source";
rev = "280543933507839201547f831280faac614d0514"; revCount = 12454; shortRev = "2805439"; submod
ules = false; }
#####################
Only 2 uses of `allRefs` in the entirety of Nixpkgs: TODO i don't think it matters if used or not; except if `builtins.fetchGit` used "improperly" -> see issue below
0 [07:38:22] ag 'allRefs' .
pkgs/development/tools/yarn2nix-moretea/yarn2nix/lib/generateNix.js
54: allRefs = true;
pkgs/development/tools/poetry2nix/poetry2nix/mk-poetry-dep.nix
179: allRefs = true;
#####################
https://github.com/NixOS/nix/issues/5128
but the error makes sense as an SHA-1 hash **is not** a valid reference
nix-repl> builtins.fetchGit { ref = "db1442a0556c2b133627ffebf455a78a1ced64b9"; rev = "db1442a055
6c2b133627ffebf455a78a1ced64b9"; url = "https://github.com/tmcw/leftpad"; }
fetching Git repository 'https://github.com/tmcw/leftpad'fatal: couldn't find remote ref refs/hea
ds/db1442a0556c2b133627ffebf455a78a1ced64b9
error: program 'git' failed with exit code 128
nix-repl> builtins.fetchGit { ref = "db1442a0556c2b133627ffebf455a78a1ced64b9"; rev = "db1442a055
6c2b133627ffebf455a78a1ced64b9"; url = "https://github.com/tmcw/leftpad"; allRefs = true; }
warning: could not update mtime for file '/home/toraritte/.cache/nix/gitv3/0240dfgnkwmgqs7sma8rns
8wlwxiv40b1lddl2sg2i0hnw7ym5c0/refs/heads/db1442a0556c2b133627ffebf455a78a1ced64b9': No such file
or directory
{ lastModified = 1493781506; lastModifiedDate = "20170503031826"; narHash = "sha256-0DbZHwAdvEUiH
o3brZyyxw0WdNQOsQwGZZz4tboN3v8="; outPath = "/nix/store/8frq54wwgi63wqgkc7p6yrcljlx4zwzh-source";
rev = "db1442a0556c2b133627ffebf455a78a1ced64b9"; revCount = 5; shortRev = "db1442a"; submodules
= false; }
nix-repl>
nix-repl>
nix-repl> builtins.fetchGit { ref = "db1442a0556c2b133627ffebf455a78a1ced64b9"; url = "https://gi
fetching Git repository 'https://github.com/tmcw/leftpad'fatal: couldn't find remote ref refs/hea
ds/db1442a0556c2b133627ffebf455a78a1ced64b9
error: program 'git' failed with exit code 128
nix-repl> builtins.fetchGit { rev = "db1442a0556c2b133627ffebf455a78a1ced64b9"; url = "https://gi
{ lastModified = 1493781506; lastModifiedDate = "20170503031826"; narHash = "sha256-0DbZHwAdvEUiH
o3brZyyxw0WdNQOsQwGZZz4tboN3v8="; outPath = "/nix/store/8frq54wwgi63wqgkc7p6yrcljlx4zwzh-source";
rev = "db1442a0556c2b133627ffebf455a78a1ced64b9"; revCount = 5; shortRev = "db1442a"; submodules
= false; }
