0

Git LFS doesn't work anymore with Yocto: git doesn't fetch "large files" since we upgraded our tools to these versions:

Yocto kirkstone gitlab 16.7.3 and gitlab 16.8 git version 2.34.1 git-lfs/3.0.2 (GitHub; linux amd64; go 1.18.1) Kubuntu 22.04: Linux linux-dev 5.15.0-91-generic #101-Ubuntu SMP Tue Nov 14 13:30:08 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux

When I execute the Yocto command "bitbake tep2020ihm", I get multiple "Smudge" errors, like that one:

ERROR: tep2020ihm-1.22-r0 do_unpack: Bitbake Fetcher Error: FetchError( <...>) failed with exit code 128, output:\nDownloading 3rd-party/aghaf/lib/aghaf.dll (6.6 MB)\nError downloading object: 3rd-party/aghaf/lib/aghaf.dll (1d6198d): Smudge error: Error downloading 3rd-party/aghaf/lib/aghaf.dll

When I'm cloning the project manually (to another directory on the dev server, by typing manually the command "git clone https://...", it works, including the large files. When the Yocto fetcher does it, it does not work (smudge errors for large files).

If I put the mention "lfs=0" in the SRC_URI variable of the .bb file (the Yocto recipe for my project), the fetcher works, but instead of fetching the file's content, it fetches the "LFS blob" instead:

version https://git-lfs.github.com/spec/v1 oid sha256:dba4dab327a3b420509ac9f466acde6c657698ccb1854522d20d3cfb35a493c1 size 173735936

I'm producing below the result of the command "git lfs logs last" in case it may clarify a point that I forgot to mention:

git-lfs/3.0.2 (GitHub; linux amd64; go 1.18.1) git version 2.34.1 $ git-lfs filter-process Error downloading object: 3rd-party/aghaf/lib/aghaf.dll (1d6198d): Smudge error: Error downloading 3rd-party/aghaf/lib/aghaf.dll (1d6198d4a0a31d3425d50901619f3b223e15d8f328c5a554d6150a0cb00fb5fd): batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution batch response: Post "https://jlsabatier:***@gitlab.annecyelectronique.com/annecyelectronique/pfd2020/tep2020_ihm.git/info/lfs/objects/batch": dial tcp: lookup gitlab.annecyelectronique.com: Temporary failure in name resolution Error downloading 3rd-party/aghaf/lib/aghaf.dll (1d6198d4a0a31d3425d50901619f3b223e15d8f328c5a554d6150a0cb00fb5fd) github.com/git-lfs/git-lfs/errors.newWrappedError github.com/git-lfs/git-lfs/errors/types.go:225 github.com/git-lfs/git-lfs/errors.Wrapf github.com/git-lfs/git-lfs/errors/errors.go:85 github.com/git-lfs/git-lfs/lfs.(*GitFilter).downloadFile github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:119 github.com/git-lfs/git-lfs/lfs.(*GitFilter).Smudge github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:78 github.com/git-lfs/git-lfs/commands.smudge github.com/git-lfs/git-lfs/commands/command_smudge.go:127 github.com/git-lfs/git-lfs/commands.filterCommand github.com/git-lfs/git-lfs/commands/command_filter_process.go:122 github.com/spf13/cobra.(*Command).execute github.com/spf13/cobra/command.go:860 github.com/spf13/cobra.(*Command).ExecuteC github.com/spf13/cobra/command.go:974 github.com/spf13/cobra.(*Command).Execute github.com/spf13/cobra/command.go:902 github.com/git-lfs/git-lfs/commands.Run github.com/git-lfs/git-lfs/commands/run.go:105 main.main github.com/git-lfs/git-lfs/git-lfs.go:33 runtime.main runtime/proc.go:250 runtime.goexit runtime/asm_amd64.s:1571 Smudge error github.com/git-lfs/git-lfs/errors.newWrappedError github.com/git-lfs/git-lfs/errors/types.go:225 github.com/git-lfs/git-lfs/errors.NewSmudgeError github.com/git-lfs/git-lfs/errors/types.go:311 github.com/git-lfs/git-lfs/lfs.(*GitFilter).Smudge github.com/git-lfs/git-lfs/lfs/gitfilter_smudge.go:87 github.com/git-lfs/git-lfs/commands.smudge github.com/git-lfs/git-lfs/commands/command_smudge.go:127 github.com/git-lfs/git-lfs/commands.filterCommand github.com/git-lfs/git-lfs/commands/command_filter_process.go:122 github.com/spf13/cobra.(*Command).execute github.com/spf13/cobra/command.go:860 github.com/spf13/cobra.(*Command).ExecuteC github.com/spf13/cobra/command.go:974 github.com/spf13/cobra.(*Command).Execute github.com/spf13/cobra/command.go:902 github.com/git-lfs/git-lfs/commands.Run github.com/git-lfs/git-lfs/commands/run.go:105 main.main github.com/git-lfs/git-lfs/git-lfs.go:33 runtime.main runtime/proc.go:250 runtime.goexit runtime/asm_amd64.s:1571 Current time in UTC: 2024-01-18 17:17:38 ENV: LocalWorkingDir=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0/git LocalGitDir=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0/git/.git LocalGitStorageDir=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0/git/.git LocalMediaDir=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0/git/.git/lfs/objects LocalReferenceDirs= TempDir=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0/git/.git/lfs/tmp ConcurrentTransfers=8 TusTransfers=false BasicTransfersOnly=false SkipDownloadErrors=false FetchRecentAlways=false FetchRecentRefsDays=7 FetchRecentCommitsDays=0 FetchRecentRefsIncludeRemotes=true PruneOffsetDays=3 PruneVerifyRemoteAlways=false PruneRemoteName=origin LfsStorageDir=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0/git/.git/lfs AccessDownload=none AccessUpload=none DownloadTransfers=basic,lfs-standalone-file,ssh UploadTransfers=basic,lfs-standalone-file,ssh GIT_EXEC_PATH=/usr/lib/git-core GIT_CEILING_DIRECTORIES=/home/jlsabatier/var-fslc-yocto-kirkstone/build_tep/tmp/work/cortexa7t2hf-neon-fslc-linux-gnueabi/tep2020ihm/1.22-r0 GIT_PREFIX= GIT_CONFIG_PARAMETERS='core.fsyncobjectfiles'='0' 'gc.autoDetach'='false' 'core.pager'='cat' Client IP addresses:
Improve this question

1 Answer 1

Reset to default
1

The message “Temporary failure in name resolution” means that you have some sort of DNS problem when trying to resolve the hostname gitlab.annecyelectronique.com. If you fix that, it's likely things will work again.

This can be for a variety of reasons, including a broken local resolver, broken recursive nameserver on the local network, broken authoritative DNS servers (including DNSSEC problems), or network problems. Git LFS is written in Go, and by default, Go doesn't use libc to resolve names, so if you have this name configured in something like mdns or other non-DNS systems, you may need to set GODEBUG=netdns=cgo in the environment to get it working.

Overall, though, this is almost certainly some sort of misconfiguration somewhere and not a bug in Git LFS.

Improve this answer
1
  • Thanks for your reply. After more investigation, it seems that Yocto has access to the network only during the fetch phase, and not during the build phase. However, git lfs uses the clean filter during the fetch phase (i.e. not pulling the full content of the LFS files), and when it tries to pull the full content (using the smudge filter) at compilation time, it cannot. After more investigation, it seems that there is an incompatibility in the SRC_URI between "nobranch=1" and "lfs=1". If I give the name of the branch (e.g. "branch=master;lfs=1" then it works, with "nobranch=1;lfs=1" it fails Commented Feb 5, 2024 at 12:31

You must log in to answer this question.

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.