Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

runtime/cgo: unrecognized relocation with binaries shipped in go1.12.2 #31293

Closed
jayconrod opened this issue Apr 5, 2019 · 27 comments · Fixed by zalando/skipper#1025
Closed
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Linux
Milestone

Comments

@jayconrod
Copy link
Contributor

What version of Go are you using (go version)?

go1.12.2

This specifically affects the binary distribution downloaded from golang.org. Bootstrapping 1.12.2 using 1.12.1 did not have this problem.

Does this issue reproduce with the latest release?

Yes

What operating system and processor architecture are you using (go env)?

go env Output
$ go env
GOARCH="amd64"
GOBIN=""
GOCACHE="/home/bazel/.cache/go-build"
GOEXE=""
GOFLAGS=""
GOHOSTARCH="amd64"
GOHOSTOS="linux"
GOOS="linux"
GOPATH="/home/bazel/go"
GOPROXY=""
GORACE=""
GOROOT="/home/bazel/go1.12.2"
GOTMPDIR=""
GOTOOLDIR="/home/bazel/go1.12.2/pkg/tool/linux_amd64"
GCCGO="gccgo"
CC="gcc"
CXX="g++"
CGO_ENABLED="1"
GOMOD=""
CGO_CFLAGS="-g -O2"
CGO_CPPFLAGS=""
CGO_CXXFLAGS="-g -O2"
CGO_FFLAGS="-g -O2"
CGO_LDFLAGS="-g -O2"
PKG_CONFIG="pkg-config"
GOGCCFLAGS="-fPIC -m64 -pthread -fmessage-length=0 -fdebug-prefix-map=/tmp/go-build845022667=/tmp/go-build -gno-record-gcc-switches"

This error is most easily reproduced with the gcr.io/bazel-untrusted/ubuntu1404:java8 docker image that Bazel uses for testing. Sorry, it's pretty big.

The important part of this is that it's on ubuntu14.04, which has an older linker.

bazel@269f5850547f:~$ /usr/bin/ld -v
GNU ld (GNU Binutils for Ubuntu) 2.24
bazel@269f5850547f:~$ /usr/bin/ld.gold -v
GNU gold (GNU Binutils for Ubuntu 2.24) 1.11
bazel@269f5850547f:~$ /usr/bin/gcc -v
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
COLLECT_LTO_WRAPPER=/usr/lib/gcc/x86_64-linux-gnu/4.8/lto-wrapper
Target: x86_64-linux-gnu
Configured with: ../src/configure -v --with-pkgversion='Ubuntu 4.8.4-2ubuntu1~14.04.4' --with-bugurl=file:///usr/share/doc/gcc-4.8/README.Bugs --enable-languages=c,c++,java,go,d,fortran,objc,obj-c++ --prefix=/usr --program-suffix=-4.8 --enable-shared --enable-linker-build-id --libexecdir=/usr/lib --without-included-gettext --enable-threads=posix --with-gxx-include-dir=/usr/include/c++/4.8 --libdir=/usr/lib --enable-nls --with-sysroot=/ --enable-clocale=gnu --enable-libstdcxx-debug --enable-libstdcxx-time=yes --enable-gnu-unique-object --disable-libmudflap --enable-plugin --with-system-zlib --disable-browser-plugin --enable-java-awt=gtk --enable-gtk-cairo --with-java-home=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64/jre --enable-java-home --with-jvm-root-dir=/usr/lib/jvm/java-1.5.0-gcj-4.8-amd64 --with-jvm-jar-dir=/usr/lib/jvm-exports/java-1.5.0-gcj-4.8-amd64 --with-arch-directory=amd64 --with-ecj-jar=/usr/share/java/eclipse-ecj.jar --enable-objc-gc --enable-multiarch --disable-werror --with-arch-32=i686 --with-abi=m64 --with-multilib-list=m32,m64,mx32 --with-tune=generic --enable-checking=release --build=x86_64-linux-gnu --host=x86_64-linux-gnu --target=x86_64-linux-gnu
Thread model: posix
gcc version 4.8.4 (Ubuntu 4.8.4-2ubuntu1~14.04.4) 

What did you do?

Download go1.12.2

Try to build the program below:

package main

// const int x = 42;
import "C"

import "fmt"

func main() {
  fmt.Println(int(C.x))
}
~/go1.12.2/bin/go build hello.go

What did you expect to see?

Successful build.

What did you see instead?

# command-line-arguments
/home/bazel/go1.12.2/pkg/tool/linux_amd64/link: running gcc failed: exit status 1
/usr/bin/ld: /tmp/go-link-648962525/000005.o: unrecognized relocation (0x2a) in section `.text'
/usr/bin/ld: final link failed: Bad value
collect2: error: ld returned 1 exit status

I'm seeing similar error when Bazel links binaries with cgo code.

external/go_sdk/pkg/tool/linux_amd64/link: running /usr/bin/gcc failed: exit status /usr/bin/ld.gold: error: /tmp/go-link-028957856/000005.o: unsupported reloc 42 against global symbol stderr
/usr/bin/ld.gold: error: /tmp/go-link-028957856/000006.o: unsupported reloc 42 against global symbol stderr
/usr/bin/ld.gold: error: /tmp/go-link-028957856/000007.o: unsupported reloc 42 against global symbol x_cgo_threadentry
/usr/bin/ld.gold: error: /tmp/go-link-028957856/000007.o: unsupported reloc 42 against global symbol x_cgo_inittls
/usr/bin/ld.gold: error: /tmp/go-link-028957856/000012.o: unsupported reloc 42 against global symbol stderr
gcc_fatalf.c:17: error: unsupported reloc 42
gcc_libinit.c:29: error: unsupported reloc 42
gcc_linux_amd64.c:86: error: unsupported reloc 42
gcc_linux_amd64.c:52: error: unsupported reloc 42
gcc_util.c:18: error: unsupported reloc 42
collect2: error: ld returned 1 exit status

Other thoughts

@ianlancetaylor told me that reloc 42 is R_X86_64_REX_GOTPCRELX. Support for this was added to gold in 2015, so it's presumably not supported by ubuntu14.04. Support was added to the Go linker for #13114. The Go toolchain should not be emitting these relocs. We may be incorporating .o files produced by a C compiler that emits these on the system that produces the binary distribution.

I was able to work around this by bootstrapping the Go 1.12.2 toolchain locally. I think the C compiler installed on this image does not emit these relocs.

@jayconrod jayconrod added NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Linux labels Apr 5, 2019
@jayconrod
Copy link
Contributor Author

cc @ianlancetaylor @dmitshur

@ianlancetaylor ianlancetaylor added this to the Go1.13 milestone Apr 5, 2019
@ianlancetaylor
Copy link
Contributor

The relocation appears in runtime/cgo for C code that refers to global variables.

The 1.12.2 binary distribution must have been built with a newer version of GCC than the 1.12.1 binary distribution. The only reasonable fix is to build it with an older version of GCC.

I'm not sure whether we should do anything about this. As a general rule, if you want the binary distribution to run on release N, you need to build it on release N, not a later release. This is one example of that, but there are many others. I don't remember whether we've ever documented anything about what systems the binary distributions run on.

CC @bradfitz @andybons

@bradfitz
Copy link
Contributor

bradfitz commented Apr 6, 2019

We updated our linux-amd64 builder image from Debian Jessie to Stretch. That's surely the cause. We obviously didn't consider that it'd have any effect on release binaries. We also don't version our builder images and our release automation doesn't use different builders as a function of branch either. Maybe it should. We never thought about it.

@bradfitz
Copy link
Contributor

bradfitz commented Apr 6, 2019

Actually we do version our builder images a bit at a broad level. But we do update a number of them "in-place" without changing their names and we definitely don't select release builders as a function of branch.

@sbinet
Copy link
Member

sbinet commented Apr 6, 2019

just to note this is also the case for the go1.11.7 builds:

@karalabe
Copy link
Contributor

karalabe commented Apr 6, 2019

@jayconrod
Copy link
Contributor Author

For Travis CI errors, if you're willing to upgrade to Ubuntu 16.04, you can do so by changing dist: trusty to dist: xenial in your .travis.yml file.

@aronatkins
Copy link

On Ubuntu 14.04:

$ ld -v
GNU ld (GNU Binutils for Ubuntu) 2.24

Issue #31336 is similar. We see this failure on Ubuntu 14.04 and the #31336 failure on CentOS 6 build hosts.

@ianlancetaylor
Copy link
Contributor

This was reported again on #31336. We may need to plan to fix this.

sbinet added a commit to sbinet-hep/hep that referenced this issue Apr 8, 2019
Migrate to xenial to workaround golang/go#31293
@gopherbot
Copy link

Change https://golang.org/cl/171121 mentions this issue: dashboard, cmd/release: use Jessie for release builds

gopherbot pushed a commit to golang/build that referenced this issue Apr 8, 2019
Drop linux-amd64-stretch (which is redundant with linux-amd64, which
is now Stretch since CL 170282) and add linux-amd64-jessie, and use
Jessie in cmd/release for Linux releases)

Updates golang/go#31293

Change-Id: I21b7a43a24385d5976296d0e5f1c6009cbc18b05
Reviewed-on: https://go-review.googlesource.com/c/build/+/171121
Reviewed-by: Andrew Bonventre <andybons@golang.org>
@andybons
Copy link
Member

andybons commented Apr 8, 2019

@gopherbot please backport to 1.11 and 1.12; the fix doesn't need to be cherry-picked, but it will be used to track the issue in the milestone.

@gopherbot
Copy link

Backport issue(s) opened: #31346 (for 1.11), #31347 (for 1.12).

Remember to create the cherry-pick CL(s) as soon as the patch is submitted to master, according to https://golang.org/wiki/MinorReleases.

@andybons
Copy link
Member

andybons commented Apr 8, 2019

1.12.3 and 1.11.8 have been built using Jessie and released. This should resolve the issue. Please let us know if there are any other problems and sorry for the trouble.

Thanks.

@andybons andybons closed this as completed Apr 8, 2019
@mwhudson
Copy link
Contributor

mwhudson commented Apr 8, 2019

FWIW, this particular issue can be mitigated by passing "-Wa,-mrelax-relocations=no" when compiling. But if you do that, unless you set CGO_FLAGS to this every time you build, the new (ish) code in the go tool considers the distributed runtime/cgo.a stale and doesn't use it. In fact, given the build cache I'm not sure there is any real need to distribute runtime/cgo.a at all -- I stopped for my snap and as far as I can tell noone has noticed...

agis added a commit to skroutz/downloader that referenced this issue Apr 12, 2019
agis added a commit to skroutz/downloader that referenced this issue Apr 12, 2019
thaJeztah added a commit to thaJeztah/golang-cross that referenced this issue Apr 13, 2019
go1.11.9 (released 2019/04/11) fixes an issue where using the prebuilt
binary releases on older versions of GNU/Linux led to failures when linking
programs that used cgo. Only Linux users who hit this issue need to update.

See golang/go#31293 for details

Full diff: golang/go@go1.11.8...go1.11.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
thaJeztah added a commit to thaJeztah/golang-cross that referenced this issue Apr 13, 2019
go1.12.4 (released 2019/04/11) fixes an issue where using the prebuilt
binary releases on older versions of GNU/Linux led to failures when linking
programs that used cgo. Only Linux users who hit this issue need to update.

See golang/go#31293 for details

Full diff: golang/go@go1.12.3...go1.12.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
thaJeztah added a commit to thaJeztah/cli that referenced this issue Apr 14, 2019
go1.12.4 (released 2019/04/11) fixes an issue where using the prebuilt
binary releases on older versions of GNU/Linux led to failures when linking
programs that used cgo. Only Linux users who hit this issue need to update.

See golang/go#31293 for details

Full diff: golang/go@go1.12.3...go1.12.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
thaJeztah added a commit to thaJeztah/cli that referenced this issue Apr 14, 2019
go1.11.9 (released 2019/04/11) fixes an issue where using the prebuilt
binary releases on older versions of GNU/Linux led to failures when linking
programs that used cgo. Only Linux users who hit this issue need to update.

See golang/go#31293 for details

Full diff: golang/go@go1.11.8...go1.11.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
docker-jenkins pushed a commit to docker/docker-ce that referenced this issue Apr 18, 2019
go1.12.4 (released 2019/04/11) fixes an issue where using the prebuilt
binary releases on older versions of GNU/Linux led to failures when linking
programs that used cgo. Only Linux users who hit this issue need to update.

See golang/go#31293 for details

Full diff: golang/go@go1.12.3...go1.12.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: cf47bb2cc2fecf5b0ccd32a5c10bc738d73bab49
Component: cli
tiborvass pushed a commit to tiborvass/cli that referenced this issue Apr 23, 2019
go1.12.4 (released 2019/04/11) fixes an issue where using the prebuilt
binary releases on older versions of GNU/Linux led to failures when linking
programs that used cgo. Only Linux users who hit this issue need to update.

See golang/go#31293 for details

Full diff: golang/go@go1.12.3...go1.12.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
thaJeztah added a commit to thaJeztah/cli that referenced this issue May 20, 2019
go1.11.9 (released 2019/04/11) fixes an issue where using the prebuilt
binary releases on older versions of GNU/Linux led to failures when linking
programs that used cgo. Only Linux users who hit this issue need to update.

See golang/go#31293 for details

Full diff: golang/go@go1.11.8...go1.11.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
thaJeztah added a commit to thaJeztah/cli that referenced this issue Jun 14, 2019
go1.11.9 (released 2019/04/11) fixes an issue where using the prebuilt
binary releases on older versions of GNU/Linux led to failures when linking
programs that used cgo. Only Linux users who hit this issue need to update.

See golang/go#31293 for details

Full diff: golang/go@go1.11.8...go1.11.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
thaJeztah added a commit to thaJeztah/docker that referenced this issue Jun 20, 2019
Full diff: golang/go@go1.11.5...go1.11.9

go1.11.6 (released 2019/03/14) includes fixes to cgo, the compiler, linker,
runtime, go command, and the crypto/x509, encoding/json, net, and net/url
packages. See the Go 1.11.6 milestone on our issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.11.6

go1.11.7 (released 2019/04/05) includes fixes to the runtime and the net
packages. See the Go 1.11.7 milestone on our issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.11.7

go1.11.8 (released 2019/04/08) was accidentally released without its
intended fix. It is identical to go1.11.7, except for its version number.
The intended fix is in go1.11.9.

go1.11.9 (released 2019/04/11) fixes an issue where using the prebuilt
binary releases on older versions of GNU/Linux led to failures when linking
programs that used cgo. Only Linux users who hit this issue need to update.

See golang/go#31293 for details

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
flga added a commit to flga/freetype2 that referenced this issue Jul 19, 2019
docker-jenkins pushed a commit to docker/docker-ce that referenced this issue Jul 24, 2019
go1.11.9 (released 2019/04/11) fixes an issue where using the prebuilt
binary releases on older versions of GNU/Linux led to failures when linking
programs that used cgo. Only Linux users who hit this issue need to update.

See golang/go#31293 for details

Full diff: golang/go@go1.11.8...go1.11.9

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 1b3ed845356771307f8e6cc79923d94fa1155b17
Component: cli
docker-jenkins pushed a commit to docker/docker-ce that referenced this issue Jul 24, 2019
Full diff: golang/go@go1.11.5...go1.11.9

go1.11.6 (released 2019/03/14) includes fixes to cgo, the compiler, linker,
runtime, go command, and the crypto/x509, encoding/json, net, and net/url
packages. See the Go 1.11.6 milestone on our issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.11.6

go1.11.7 (released 2019/04/05) includes fixes to the runtime and the net
packages. See the Go 1.11.7 milestone on our issue tracker for details.

https://github.com/golang/go/issues?q=milestone%3AGo1.11.7

go1.11.8 (released 2019/04/08) was accidentally released without its
intended fix. It is identical to go1.11.7, except for its version number.
The intended fix is in go1.11.9.

go1.11.9 (released 2019/04/11) fixes an issue where using the prebuilt
binary releases on older versions of GNU/Linux led to failures when linking
programs that used cgo. Only Linux users who hit this issue need to update.

See golang/go#31293 for details

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
Upstream-commit: 2cc1df06ee264ab5d1769bd4aae9e417feda9402
Component: engine
kolyshkin pushed a commit to kolyshkin/cli that referenced this issue Sep 28, 2019
go1.12.4 (released 2019/04/11) fixes an issue where using the prebuilt
binary releases on older versions of GNU/Linux led to failures when linking
programs that used cgo. Only Linux users who hit this issue need to update.

See golang/go#31293 for details

Full diff: golang/go@go1.12.3...go1.12.4

Signed-off-by: Sebastiaan van Stijn <github@gone.nl>
(cherry picked from commit cf47bb2)
Signed-off-by: Kir Kolyshkin <kolyshkin@gmail.com>
@golang golang locked and limited conversation to collaborators Apr 11, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
FrozenDueToAge NeedsInvestigation Someone must examine and confirm this is a valid issue and not a duplicate of an existing one. OS-Linux
Projects
None yet
Development

Successfully merging a pull request may close this issue.