| 
									
										
										
										
											2022-06-23 08:55:05 -04:00
										 |  |  | #!/bin/bash -e
 | 
					
						
							| 
									
										
										
										
											2020-06-12 01:51:45 -04:00
										 |  |  | # | 
					
						
							|  |  |  | # OSS-Fuzz build script. See: | 
					
						
							|  |  |  | # https://google.github.io/oss-fuzz/getting-started/new-project-guide/#buildsh | 
					
						
							|  |  |  | # | 
					
						
							|  |  |  | # The file is consumed by: | 
					
						
							|  |  |  | # https://github.com/google/oss-fuzz/blob/master/projects/qemu/Dockerfiles | 
					
						
							|  |  |  | # | 
					
						
							|  |  |  | # This code is licensed under the GPL version 2 or later.  See | 
					
						
							|  |  |  | # the COPYING file in the top-level directory. | 
					
						
							|  |  |  | # | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | # build project | 
					
						
							|  |  |  | # e.g. | 
					
						
							|  |  |  | # ./autogen.sh | 
					
						
							|  |  |  | # ./configure | 
					
						
							|  |  |  | # make -j$(nproc) all | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | # build fuzzers | 
					
						
							|  |  |  | # e.g. | 
					
						
							|  |  |  | # $CXX $CXXFLAGS -std=c++11 -Iinclude \ | 
					
						
							|  |  |  | #     /path/to/name_of_fuzzer.cc -o $OUT/name_of_fuzzer \ | 
					
						
							| 
									
										
										
										
											2020-08-09 17:17:00 +02:00
										 |  |  | #     -fsanitize=fuzzer /path/to/library.a | 
					
						
							| 
									
										
										
										
											2020-06-12 01:51:45 -04:00
										 |  |  | 
 | 
					
						
							|  |  |  | fatal () { | 
					
						
							|  |  |  |     echo "Error : ${*}, exiting." | 
					
						
							|  |  |  |     exit 1 | 
					
						
							|  |  |  | } | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | OSS_FUZZ_BUILD_DIR="./build-oss-fuzz/" | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | # There seems to be a bug in clang-11 (used for builds on oss-fuzz) : | 
					
						
							|  |  |  | #   accel/tcg/cputlb.o: In function `load_memop': | 
					
						
							|  |  |  | #   accel/tcg/cputlb.c:1505: undefined reference to `qemu_build_not_reached' | 
					
						
							|  |  |  | # | 
					
						
							|  |  |  | # When building with optimization, the compiler is expected to prove that the | 
					
						
							|  |  |  | # statement cannot be reached, and remove it. For some reason clang-11 doesn't | 
					
						
							|  |  |  | # remove it, resulting in an unresolved reference to qemu_build_not_reached | 
					
						
							|  |  |  | # Undefine the __OPTIMIZE__ macro which compiler.h relies on to choose whether | 
					
						
							|  |  |  | # to " #define qemu_build_not_reached()  g_assert_not_reached() " | 
					
						
							|  |  |  | EXTRA_CFLAGS="$CFLAGS -U __OPTIMIZE__" | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | if ! { [ -e "./COPYING" ] && | 
					
						
							|  |  |  |    [ -e "./MAINTAINERS" ] && | 
					
						
							|  |  |  |    [ -e "./Makefile" ] && | 
					
						
							| 
									
										
										
										
											2023-10-04 11:06:28 +02:00
										 |  |  |    [ -d "./docs" ] && | 
					
						
							| 
									
										
										
										
											2020-06-12 01:51:45 -04:00
										 |  |  |    [ -e "./VERSION" ] && | 
					
						
							| 
									
										
										
										
											2023-10-04 11:06:28 +02:00
										 |  |  |    [ -d "./linux-user" ] && | 
					
						
							|  |  |  |    [ -d "./system" ];} ; then | 
					
						
							| 
									
										
										
										
											2020-06-12 01:51:45 -04:00
										 |  |  |     fatal "Please run the script from the top of the QEMU tree" | 
					
						
							|  |  |  | fi | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | mkdir -p $OSS_FUZZ_BUILD_DIR || fatal "mkdir $OSS_FUZZ_BUILD_DIR failed" | 
					
						
							|  |  |  | cd $OSS_FUZZ_BUILD_DIR || fatal "cd $OSS_FUZZ_BUILD_DIR failed" | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | if [ -z ${OUT+x} ]; then | 
					
						
							|  |  |  |     DEST_DIR=$(realpath "./DEST_DIR") | 
					
						
							|  |  |  | else | 
					
						
							|  |  |  |     DEST_DIR=$OUT | 
					
						
							|  |  |  | fi | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | mkdir -p "$DEST_DIR/lib/"  # Copy the shared libraries here | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | # Build once to get the list of dynamic lib paths, and copy them over | 
					
						
							| 
									
										
										
										
											2020-08-09 17:17:00 +02:00
										 |  |  | ../configure --disable-werror --cc="$CC" --cxx="$CXX" --enable-fuzzing \
 | 
					
						
							| 
									
										
										
										
											2022-06-24 23:50:38 +09:00
										 |  |  |     --prefix="/opt/qemu-oss-fuzz" \
 | 
					
						
							| 
									
										
										
										
											2020-07-17 08:57:42 +02:00
										 |  |  |     --extra-cflags="$EXTRA_CFLAGS" --target-list="i386-softmmu" | 
					
						
							| 
									
										
										
										
											2020-06-12 01:51:45 -04:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2020-08-03 17:04:25 +02:00
										 |  |  | if ! make "-j$(nproc)" qemu-fuzz-i386; then | 
					
						
							| 
									
										
										
										
											2020-06-12 01:51:45 -04:00
										 |  |  |     fatal "Build failed. Please specify a compiler with fuzzing support"\
 | 
					
						
							| 
									
										
										
										
											2020-09-17 15:50:23 +08:00
										 |  |  |           "using the \$CC and \$CXX environment variables"\
 | 
					
						
							| 
									
										
										
										
											2020-06-12 01:51:45 -04:00
										 |  |  |           "\nFor example: CC=clang CXX=clang++ $0" | 
					
						
							|  |  |  | fi | 
					
						
							|  |  |  | 
 | 
					
						
							| 
									
										
										
										
											2021-08-09 07:16:21 -04:00
										 |  |  | if [ "$GITLAB_CI" != "true" ]; then | 
					
						
							|  |  |  |     for i in $(ldd ./qemu-fuzz-i386 | cut -f3 -d' '); do | 
					
						
							|  |  |  |         cp "$i" "$DEST_DIR/lib/" | 
					
						
							|  |  |  |     done | 
					
						
							|  |  |  |     rm qemu-fuzz-i386 | 
					
						
							| 
									
										
										
										
											2020-06-12 01:51:45 -04:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2021-08-09 07:16:21 -04:00
										 |  |  |     # Build a second time to build the final binary with correct rpath | 
					
						
							|  |  |  |     ../configure --disable-werror --cc="$CC" --cxx="$CXX" --enable-fuzzing \
 | 
					
						
							| 
									
										
										
										
											2022-06-24 23:50:38 +09:00
										 |  |  |         --prefix="/opt/qemu-oss-fuzz" \
 | 
					
						
							| 
									
										
										
										
											2021-08-09 07:16:21 -04:00
										 |  |  |         --extra-cflags="$EXTRA_CFLAGS" --extra-ldflags="-Wl,-rpath,\$ORIGIN/lib" \
 | 
					
						
							|  |  |  |         --target-list="i386-softmmu" | 
					
						
							|  |  |  |     make "-j$(nproc)" qemu-fuzz-i386 V=1 | 
					
						
							|  |  |  | fi | 
					
						
							| 
									
										
										
										
											2020-06-12 01:51:45 -04:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2022-07-20 10:40:06 +02:00
										 |  |  | # Place data files in the preinstall tree | 
					
						
							| 
									
										
										
										
											2022-06-24 23:50:38 +09:00
										 |  |  | make install DESTDIR=$DEST_DIR/qemu-bundle | 
					
						
							| 
									
										
										
										
											2022-07-20 10:40:06 +02:00
										 |  |  | rm -rf $DEST_DIR/qemu-bundle/opt/qemu-oss-fuzz/bin | 
					
						
							|  |  |  | rm -rf $DEST_DIR/qemu-bundle/opt/qemu-oss-fuzz/libexec | 
					
						
							| 
									
										
										
										
											2020-06-12 01:51:45 -04:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2024-05-27 10:59:58 -04:00
										 |  |  | export ASAN_OPTIONS=detect_leaks=0 | 
					
						
							| 
									
										
										
										
											2022-07-20 14:09:46 -04:00
										 |  |  | targets=$(./qemu-fuzz-i386 | grep generic-fuzz | awk '$1 ~ /\*/  {print $2}') | 
					
						
							| 
									
										
											  
											
												scripts/oss-fuzz: give all fuzzers -target names
We switched to hardlinks in
a942f64cc4 ("scripts/oss-fuzz: use hardlinks instead of copying")
The motivation was to conserve space (50 fuzzers built with ASAN, can
weigh close to 9 GB).
Unfortunately, OSS-Fuzz (partially) treated the underlying copy of the
fuzzer as a standalone fuzzer. To attempt to fix, we tried:
f8b8f37463 ("scripts/oss-fuzz: rename bin/qemu-fuzz-i386")
This was also not a complete fix, because though OSS-Fuzz
ignores the renamed fuzzer, the underlying ClusterFuzz, doesn't:
https://storage.googleapis.com/clusterfuzz-builds/qemu/targets.list.address
https://oss-fuzz-build-logs.storage.googleapis.com/log-9bfb55f9-1c20-4aa6-a49c-ede12864eeb2.txt
(clusterfuzz still lists qemu-fuzz-i386.base as a fuzzer)
This change keeps the hard-links, but makes them all point to a file
with a qemu-fuzz-i386-target-.. name. If we have targets, A, B, C, the
result will be:
qemu-fuzz-i386-target-A (base file)
qemu-fuzz-i386-target-B -> qemu-fuzz-i386-target-A
qemu-fuzz-i386-target-C -> qemu-fuzz-i386-target-A
The result should be that every file that looks like a fuzzer to
OSS-Fuzz/ClusterFuzz, can run as a fuzzer (we don't have a separate base
copy). Unfortunately, there is not simple way to test this locally.
In the future, it might be worth it to link the majority of QEMU in as a
shared-object (see https://github.com/google/oss-fuzz/issues/4575 )
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201108171136.160607-1-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
											
										 
											2020-11-08 12:11:36 -05:00
										 |  |  | base_copy="$DEST_DIR/qemu-fuzz-i386-target-$(echo "$targets" | head -n 1)" | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | cp "./qemu-fuzz-i386" "$base_copy" | 
					
						
							| 
									
										
										
										
											2020-10-23 11:07:45 -04:00
										 |  |  | 
 | 
					
						
							| 
									
										
										
										
											2020-06-12 01:51:45 -04:00
										 |  |  | # Run the fuzzer with no arguments, to print the help-string and get the list | 
					
						
							|  |  |  | # of available fuzz-targets. Copy over the qemu-fuzz-i386, naming it according | 
					
						
							|  |  |  | # to each available fuzz target (See 05509c8e6d fuzz: select fuzz target using | 
					
						
							|  |  |  | # executable name) | 
					
						
							| 
									
										
											  
											
												scripts/oss-fuzz: give all fuzzers -target names
We switched to hardlinks in
a942f64cc4 ("scripts/oss-fuzz: use hardlinks instead of copying")
The motivation was to conserve space (50 fuzzers built with ASAN, can
weigh close to 9 GB).
Unfortunately, OSS-Fuzz (partially) treated the underlying copy of the
fuzzer as a standalone fuzzer. To attempt to fix, we tried:
f8b8f37463 ("scripts/oss-fuzz: rename bin/qemu-fuzz-i386")
This was also not a complete fix, because though OSS-Fuzz
ignores the renamed fuzzer, the underlying ClusterFuzz, doesn't:
https://storage.googleapis.com/clusterfuzz-builds/qemu/targets.list.address
https://oss-fuzz-build-logs.storage.googleapis.com/log-9bfb55f9-1c20-4aa6-a49c-ede12864eeb2.txt
(clusterfuzz still lists qemu-fuzz-i386.base as a fuzzer)
This change keeps the hard-links, but makes them all point to a file
with a qemu-fuzz-i386-target-.. name. If we have targets, A, B, C, the
result will be:
qemu-fuzz-i386-target-A (base file)
qemu-fuzz-i386-target-B -> qemu-fuzz-i386-target-A
qemu-fuzz-i386-target-C -> qemu-fuzz-i386-target-A
The result should be that every file that looks like a fuzzer to
OSS-Fuzz/ClusterFuzz, can run as a fuzzer (we don't have a separate base
copy). Unfortunately, there is not simple way to test this locally.
In the future, it might be worth it to link the majority of QEMU in as a
shared-object (see https://github.com/google/oss-fuzz/issues/4575 )
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201108171136.160607-1-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
											
										 
											2020-11-08 12:11:36 -05:00
										 |  |  | for target in $(echo "$targets" | tail -n +2); | 
					
						
							| 
									
										
										
										
											2020-06-12 01:51:45 -04:00
										 |  |  | do | 
					
						
							| 
									
										
										
										
											2020-10-23 11:07:46 -04:00
										 |  |  |     # Ignore the generic-fuzz target, as it requires some environment variables | 
					
						
							|  |  |  |     # to be configured. We have some generic-fuzz-{pc-q35, floppy, ...} targets | 
					
						
							|  |  |  |     # that are thin wrappers around this target that set the required | 
					
						
							|  |  |  |     # environment variables according to predefined configs. | 
					
						
							| 
									
										
										
										
											2022-06-23 08:55:05 -04:00
										 |  |  |     if [[ $target == "generic-fuzz-"* ]]; then | 
					
						
							| 
									
										
											  
											
												scripts/oss-fuzz: give all fuzzers -target names
We switched to hardlinks in
a942f64cc4 ("scripts/oss-fuzz: use hardlinks instead of copying")
The motivation was to conserve space (50 fuzzers built with ASAN, can
weigh close to 9 GB).
Unfortunately, OSS-Fuzz (partially) treated the underlying copy of the
fuzzer as a standalone fuzzer. To attempt to fix, we tried:
f8b8f37463 ("scripts/oss-fuzz: rename bin/qemu-fuzz-i386")
This was also not a complete fix, because though OSS-Fuzz
ignores the renamed fuzzer, the underlying ClusterFuzz, doesn't:
https://storage.googleapis.com/clusterfuzz-builds/qemu/targets.list.address
https://oss-fuzz-build-logs.storage.googleapis.com/log-9bfb55f9-1c20-4aa6-a49c-ede12864eeb2.txt
(clusterfuzz still lists qemu-fuzz-i386.base as a fuzzer)
This change keeps the hard-links, but makes them all point to a file
with a qemu-fuzz-i386-target-.. name. If we have targets, A, B, C, the
result will be:
qemu-fuzz-i386-target-A (base file)
qemu-fuzz-i386-target-B -> qemu-fuzz-i386-target-A
qemu-fuzz-i386-target-C -> qemu-fuzz-i386-target-A
The result should be that every file that looks like a fuzzer to
OSS-Fuzz/ClusterFuzz, can run as a fuzzer (we don't have a separate base
copy). Unfortunately, there is not simple way to test this locally.
In the future, it might be worth it to link the majority of QEMU in as a
shared-object (see https://github.com/google/oss-fuzz/issues/4575 )
Signed-off-by: Alexander Bulekov <alxndr@bu.edu>
Message-Id: <20201108171136.160607-1-alxndr@bu.edu>
Signed-off-by: Thomas Huth <thuth@redhat.com>
											
										 
											2020-11-08 12:11:36 -05:00
										 |  |  |         ln  $base_copy \
 | 
					
						
							| 
									
										
										
										
											2020-10-23 11:07:46 -04:00
										 |  |  |             "$DEST_DIR/qemu-fuzz-i386-target-$target" | 
					
						
							|  |  |  |     fi | 
					
						
							| 
									
										
										
										
											2020-06-12 01:51:45 -04:00
										 |  |  | done | 
					
						
							|  |  |  | 
 | 
					
						
							|  |  |  | echo "Done. The fuzzers are located in $DEST_DIR" | 
					
						
							|  |  |  | exit 0 |