THRIFT-4351: use travis build stages to optimize build,
avoiding duplicate rebuilds of the same image, and also
allow personal docker hub repositories for private fork
builds to be optimized. Move ubsan build to artful image
because it catches more stuff and fix what was found.
THRIFT-4345: solidify docker build strategy for maximum
coverage: trusty, xenial, artful as stock as they can be
THRIFT-4344: add top level language summary markdown and
update readme with a new image on the layered architecture
THRIFT-3847: remove VERSION macro from config.h which
was causing a conflict on artful builds.
THRIFT-4359: fix haxe map/set decode when key is binary,
as a missing break statement caused it to use an int
during decode
This closes #1389
diff --git a/build/docker/README.md b/build/docker/README.md
index 2de7d68..81a4935 100644
--- a/build/docker/README.md
+++ b/build/docker/README.md
@@ -1,15 +1,40 @@
-# Apache Thrift Docker containers
+# Docker Integration #
-Docker containers used to build and test Apache Thrift for a variety of platforms.
+Due to the large number of language requirements to build Apache Thrift, docker containers are used to build and test the project on a variety of platforms to provide maximum test coverage.
-## Available Containers
+## Travis CI Integration ##
-The Travis CI (continuous integration) builds use the Ubuntu Trusty and Xenial images to maximize
-language level coverage. The other images may or may not work for all languages.
+The Travis CI scripts use the following environment variables and logic to determine their behavior.
+
+### Environment Variables ###
+
+| Variable | Default | Usage |
+| -------- | ----- | ------- |
+| `DISTRO` | `ubuntu-xenial` | Set by various build jobs in `.travis.yml` to run builds in different containers. Not intended to be set externally.|
+| `DOCKER_REPO` | `thrift` | The name of the Docker Hub repository to obtain and store docker images. |
+| `DOCKER_USER` | `apache` | The Docker Hub account name containing the repository. |
+| `DOCKER_PASS` | `<none>` | The Docker Hub account password to use when pushing new tags. |
+
+For example, the default docker image that is used in builds if no overrides are specified would be: `apache/thrift:ubuntu-xenial`
+
+If you have forked the Apache Thrift repository and you would like to use your own Docker Hub account to store thrift build images, you can use the Travis CI web interface to set the `DOCKER_USER`, `DOCKER_PASS`, and `DOCKER_REPO` variables in a secure manner.
+
+### Logic ###
+
+The Travis CI build runs in two phases - first the docker images are rebuilt for each of the three supported containers if they do not match the Dockerfile that was used to build the most recent tag. If a `DOCKER_PASS` environment variable is specified, the docker stage builds will attempt to log into Docker Hub and push the resulting tags.
+
+## Supported Containers ##
+
+The Travis CI (continuous integration) builds use the Ubuntu Trusty, Xenial, and Artful images to maximize language level coverage.
### Ubuntu
-* trusty
-* xenial (current)
+* trusty (legacy)
+* xenial (stable)
+* artful (latest)
+
+## Unsupported Containers
+
+These containers may be in various states, and may not build everything.
### CentOS
* 7.3
@@ -20,9 +45,6 @@
* stretch
* make check in lib/cpp fails due to https://svn.boost.org/trac10/ticket/12507
-## Dependencies
-* A working Docker environment. A Vagrantfile is provided which will setup an Ubuntu host and working Docker environment as well as build the Apache Thrift Docker container for testing and development.
-
## Usage
From the Apache Thrift code base root:
@@ -30,58 +52,56 @@
docker build -t thrift build/docker/ubuntu-xenial
- or
-
- docker build -t thrift build/docker/centos-7.3
-
* Run
docker run -v $(pwd):/thrift/src -it thrift /bin/bash
## Core Tool Versions per Dockerfile
-| Tool | centos-7.3 | debian-stretch | ubuntu-trusty | ubuntu-xenial | Notes |
-| :-------- | :--------- | :------------- | :------------ | :------------ | :---- |
-| ant | 1.9.2 | 1.9.9 | 1.9.3 | 1.9.6 | |
-| autoconf | 2.69 | 2.69 | 2.69 | 2.69 | |
-| automake | 1.13.4 | 1.15 | 1.14.1 | 1.15 | |
-| bison | 2.7 | 3.0.4 | 3.0.2 | 3.0.4 | |
-| boost | 1.53.0 | 1.62.0 | 1.54.0 | 1.58.0 | |
-| cmake | 3.6.3 | 3.7.2 | 3.2.2 | 3.5.1 | |
-| flex | 2.5.37 | 2.6.1 | 2.5.35 | 2.6.0 | |
-| glibc | 2.17 | 2.24 | 2.19 | 2.23 | |
-| libevent | 2.0.21 | 2.0.21 | 2.0.21 | 2.0.21 | |
-| libstdc++ | 4.8.5 | 6.3.0 | 4.8.4 | 5.4.0 | |
-| make | 3.82 | 4.1 | 3.81 | 4.1 | |
-| openssl | 1.0.1e | 1.1.0f | 1.0.1f | 1.0.2g | |
+| Tool | ubuntu-trusty | ubuntu-xenial | ubuntu-artful | Notes |
+| :-------- | :------------ | :------------ | :------------ | :---- |
+| ant | 1.9.3 | 1.9.6 | 1.9.9 | |
+| autoconf | 2.69 | 2.69 | 2.69 | |
+| automake | 1.14.1 | 1.15 | 1.15 | |
+| bison | 3.0.2 | 3.0.4 | 3.0.4 | |
+| boost | 1.54.0 | 1.58.0 | 1.63.0 | artful: stock boost 1.62.0 has problems running unit tests |
+| cmake | 3.2.2 | 3.5.1 | 3.9.1 | |
+| cppcheck | 1.61 | 1.72 | 1.80 | |
+| flex | 2.5.35 | 2.6.0 | 2.6.1 | |
+| glibc | 2.19 | 2.23 | 2.26 | |
+| libevent | 2.0.21 | 2.0.21 | 2.1 | |
+| libstdc++ | 4.8.4 | 5.4.0 | 7.2.0 | |
+| make | 3.81 | 4.1 | 4.1 | |
+| openssl | 1.0.1f | 1.0.2g | 1.0.2g | |
+| qt5 | 5.2.1 | 5.5.1 | 5.9.1 | |
-## Language Versions per Dockerfile
-| Language | centos-7.3 | debian-stretch | ubuntu-trusty | ubuntu-xenial | Notes |
-| :-------- | :--------- | :------------- | :------------ | :------------ | :---- |
-| as3 | | | | | Not in CI |
-| C++-gcc | 4.8.5 | 6.3.0 | 4.8.4 | 5.4.0 | |
-| C++-clang | 3.4.2 | 3.8.1 | 3.4 | 3.8 | |
-| C# (mono) | 4.6.2 | 4.6.2.7 | 5.2.0.224 | 5.2.0.215 | |
-| c_glib | 2.46.2 | 2.50.3 | 2.40.2 | 2.48.2 | |
-| cocoa | | | | | Not in CI |
-| d | 2.076.0 | 2.076.0 | 2.070.0 | 2.075.1 | |
-| dart | 1.24.2 | 1.24.2 | 1.24.2 | 1.24.2 | |
-| delphi | | | | | Not in CI |
-| dotnet | | | | | Not in CI |
-| erlang | 20 | 19.2 | 20 | 18.3 | |
-| go | 1.9 | 1.7.4 | 1.4.3 | 1.6.2 | |
-| haskell | 7.6.3 | 8.0.1 | 7.6.3 | 7.10.3 | |
-| haxe | | 3.2.1 | 3.2.1 | 3.2.1 | |
-| java | 1.8.0_141 | 1.8.0_141 | 1.7.0_151 | 1.8.0_131 | |
-| js | | | | | Unsure how to look for version info |
-| lua | 5.3.4 | 5.2.4 | 5.2.3 | 5.2.4 | |
-| nodejs | 6.11.1 | 8.4.0 | 4.8.4 | 7.10.1 | Node 8.5 broke copyFile and jsdoc |
-| ocaml | 4.01.0 | 4.02.3 | 4.02.3 | 4.02.3 | |
-| perl | 5.16.3 | 5.24.1 | 5.18.2 | 5.22.1 | |
-| php | 5.4.16 | 7.0.19 | 5.5.9 | 7.0.22 | |
-| python | 2.7.5 | 2.7.13 | 2.7.6 | 2.7.12 | |
-| python3 | 3.4.5 | 3.5.3 | 3.4.3 | 3.5.2 | |
-| ruby | 2.0.0p648 | 2.3.3p222 | 1.9.3p484 | 2.3.1p112 | |
-| rust | 1.17.0 | 1.14.0 | 1.17.0 | 1.15.1 | Rust is too old on stretch |
-| smalltalk | | | | | Not in CI |
-| swift | | | | | Not in CI |
+## Compiler/Language Versions per Dockerfile
+| Language | ubuntu-trusty | ubuntu-xenial | ubuntu-artful | Notes |
+| :-------- | :------------ | :------------ | :------------ | :---- |
+| as3 | | | | Not in CI |
+| C++ gcc | 4.8.4 | 5.4.0 | 7.2.0 | |
+| C++ clang | 3.4 | 3.8 | 4.0 | |
+| C# (mono) | 3.2.8.0 | 4.2.1 | 4.6.2.7 | |
+| c_glib | 2.40.2 | 2.48.2 | 2.54.0 | |
+| cocoa | | | | Not in CI |
+| d | 2.070.2 | 2.073.2 | 2.076.0 | |
+| dart | 1.20.1 | 1.24.2 | | artful: apt repo not compatible with apt 1.4? |
+| delphi | | | | Not in CI |
+| dotnet | | 2.0.0 | | Needs to be added to artful |
+| erlang | R16B03 | 18.3 | 20.0.4 | |
+| go | 1.2.1 | 1.6.2 | 1.8.3 | |
+| haskell | 7.6.3 | 7.10.3 | 8.0.2 | |
+| haxe | | 3.2.1 | 3.4.2 | disabled in trusty builds - cores on install v3.0.0, disabled in artful builds - see THRIFT-4352 |
+| java | 1.7.0_151 | 1.8.0_131 | 1.8.0_144 | |
+| js | | | | Unsure how to look for version info? |
+| lua | 5.1.5 | 5.2.4 | 5.3.3 | |
+| nodejs | | 4.2.6 | 6.11.2 | trusty has node.js 0.10.0 which is too old |
+| ocaml | | 4.02.3 | 4.04.0 | |
+| perl | 5.18.2 | 5.22.1 | 5.26.0 | |
+| php | 5.5.9 | 7.0.22 | 7.1.8 | |
+| python | 2.7.6 | 2.7.12 | 2.7.14 | |
+| python3 | 3.4.3 | 3.5.2 | 3.6.3 | |
+| ruby | 1.9.3p484 | 2.3.1p112 | 2.3.3p222 | |
+| rust | 1.15.1 | 1.15.1 | 1.18.0 | |
+| smalltalk | | | | Not in CI |
+| swift | | | | Not in CI |
diff --git a/build/docker/refresh.sh b/build/docker/refresh.sh
new file mode 100755
index 0000000..20a443b
--- /dev/null
+++ b/build/docker/refresh.sh
@@ -0,0 +1,77 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+#
+# The build has two stages: "docker" and "test"
+# The "docker" stage is meant to rebuild the docker images
+# if needed. If we cannot push that result however then
+# there is no reason to do anything.
+# The "test" stage is an actual test job. Even if the docker
+# image doesn't match what's in the repo, we still build
+# the image so the build job can run properly.
+#
+
+set -e
+
+SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
+. $SCRIPT_DIR/vars.sh
+
+function dockerfile_changed {
+ # image may not exist yet, so we have to let it fail silently:
+ docker pull $DOCKER_TAG || true
+ docker run $DOCKER_TAG bash -c 'cd .. && sha512sum Dockerfile' > .Dockerfile.sha512
+ sha512sum -c .Dockerfile.sha512
+}
+
+#
+# If this build has no DOCKER_PASS and it is in the docker stage
+# then there's no reason to do any processing because we cannot
+# push the result if the Dockerfile changed.
+#
+
+if [[ "$TRAVIS_BUILD_STAGE" == "docker" ]] && [[ -z "$DOCKER_PASS" ]]; then
+ echo Detected docker stage build and no defined DOCKER_PASS, this build job will be skipped.
+ echo Subsequent jobs in the test stage may each rebuild the docker image.
+ exit 0
+fi
+
+
+pushd ${SCRIPT_DIR}/$DISTRO
+if dockerfile_changed; then
+ echo Dockerfile has not changed. No need to rebuild.
+ exit 0
+else
+ echo Dockerfile has changed.
+fi
+popd
+
+#
+# Dockerfile has changed
+#
+
+echo Rebuilding docker image $DISTRO
+docker build --tag $DOCKER_TAG build/docker/$DISTRO
+
+if [[ ! -z "$DOCKER_PASS" ]]; then
+ echo Pushing docker image $DOCKER_TAG
+ docker login -u $DOCKER_USER -p $DOCKER_PASS
+ docker push $DOCKER_TAG
+fi
+
diff --git a/build/docker/check_unmodified.sh b/build/docker/run.sh
similarity index 60%
rename from build/docker/check_unmodified.sh
rename to build/docker/run.sh
index 9d5fa26..b54924b 100755
--- a/build/docker/check_unmodified.sh
+++ b/build/docker/run.sh
@@ -18,25 +18,13 @@
# under the License.
#
-# Download prebuilt docker image and compare Dockerfile hash values
-
-set -ex
+set -e
SCRIPT_DIR="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)"
-DISTRO=$1
-SRC_IMG=thrift/thrift-build:$DISTRO
+. $SCRIPT_DIR/vars.sh
-function try_pull {
- docker pull $SRC_IMG
- cd ${SCRIPT_DIR}/$DISTRO
- docker run $SRC_IMG bash -c 'cd .. && sha512sum Dockerfile' > .Dockerfile.sha512
- sha512sum -c .Dockerfile.sha512
-}
+printenv | sort
-if try_pull; then
- echo Dockerfile seems identical. No need to rebuild from scratch.
- docker tag thrift/thrift-build:$DISTRO thrift-build:$DISTRO
-else
- echo Either Dockerfile has changed or pull failure. Need to build brand new one.
- exit 1
-fi
+docker run --net=host -e BUILD_LIBS="$BUILD_LIBS" $BUILD_ENV -v $(pwd):/thrift/src \
+ -it $DOCKER_TAG build/docker/scripts/$SCRIPT $BUILD_ARG
+
diff --git a/build/docker/scripts/sca.sh b/build/docker/scripts/sca.sh
new file mode 100755
index 0000000..2f2fb68
--- /dev/null
+++ b/build/docker/scripts/sca.sh
@@ -0,0 +1,48 @@
+#!/bin/sh
+set -ev
+
+#
+# C/C++ static code analysis with cppcheck
+# add --error-exitcode=1 to --enable=all as soon as everything is fixed
+#
+# Python code style check with flake8
+#
+# search for TODO etc within source tree
+# some statistics about the code base
+# some info about the build machine
+
+# Compiler cppcheck (All)
+cppcheck --force --quiet --inline-suppr --enable=all -j2 compiler/cpp/src
+
+# C++ cppcheck (All)
+cppcheck --force --quiet --inline-suppr --enable=all -j2 lib/cpp/src lib/cpp/test test/cpp tutorial/cpp
+
+# C Glib cppcheck (All)
+cppcheck --force --quiet --inline-suppr --enable=all -j2 lib/c_glib/src lib/c_glib/test test/c_glib/src tutorial/c_glib
+
+# Silent error checks
+cppcheck --force --quiet --inline-suppr --error-exitcode=1 -j2 compiler/cpp/src
+cppcheck --force --quiet --inline-suppr --error-exitcode=1 -j2 lib/cpp/src lib/cpp/test test/cpp tutorial/cpp
+cppcheck --force --quiet --inline-suppr --error-exitcode=1 -j2 lib/c_glib/src lib/c_glib/test test/c_glib/src tutorial/c_glib
+
+# Python code style
+flake8 --ignore=E501 lib/py
+flake8 tutorial/py
+flake8 --ignore=E501 test/py
+flake8 test/py.twisted
+flake8 test/py.tornado
+flake8 --ignore=E501 test/test.py
+flake8 --ignore=E501 test/crossrunner
+flake8 test/features
+
+# TODO etc
+echo FIXMEs: `grep -r FIXME * | wc -l`
+echo HACKs: `grep -r HACK * | wc -l`
+echo TODOs: `grep -r TODO * | wc -l`
+
+# LoC
+sloccount .
+
+# System Info
+dpkg -l
+uname -a
diff --git a/build/docker/scripts/ubsan.sh b/build/docker/scripts/ubsan.sh
index e1e82c9..650dba0 100755
--- a/build/docker/scripts/ubsan.sh
+++ b/build/docker/scripts/ubsan.sh
@@ -1,27 +1,28 @@
#!/bin/sh
-set -ex
+set -e
-# Wraps autotools.sh, but each binary crashes if it exhibits undefined behavior. See
-# http://releases.llvm.org/3.8.0/tools/clang/docs/UndefinedBehaviorSanitizer.html
-
+# Wraps autotools.sh, but each binary crashes if it exhibits undefined behavior.
# Set the undefined behavior flags. This crashes on all undefined behavior except for
# undefined casting, aka "vptr".
-#
# TODO: fix undefined vptr behavior and turn this option back on.
-export CFLAGS="-fsanitize=undefined -fno-sanitize-recover=undefined"
-# Builds without optimization and with debugging symbols for making crash reports more
-# readable.
-export CFLAGS="${CFLAGS} -O0 -ggdb3"
+
+export CFLAGS="-fsanitize=undefined -fno-sanitize-recover=undefined -O0 -ggdb3 -fno-omit-frame-pointer"
export CXXFLAGS="${CFLAGS}"
+export LDFLAGS="-lubsan"
export UBSAN_OPTIONS=print_stacktrace=1
-# llvm-symbolizer must be on PATH, but the above installation instals a binary called
-# "llvm-symbolizer-3.8", not "llvm-symbolizer". This fixes that with a softlink in a new
-# directory.
+#
+# work around https://svn.boost.org/trac10/ticket/11632 if present
+#
+
+sed -i 's/, stream_t(rdbuf()) /, stream_t(pbase_type::member.get())/g' /usr/include/boost/format/alt_sstream.hpp
+
+# llvm-symbolizer must be on PATH to get a stack trace on error
+
CLANG_PATH="$(mktemp -d)"
trap "rm -rf ${CLANG_PATH}" EXIT
-ln -s "$(whereis llvm-symbolizer-3.8 | rev | cut -d ' ' -f 1 | rev)" \
+ln -s "$(whereis llvm-symbolizer-4.0 | rev | cut -d ' ' -f 1 | rev)" \
"${CLANG_PATH}/llvm-symbolizer"
export PATH="${CLANG_PATH}:${PATH}"
llvm-symbolizer -version
diff --git a/build/docker/ubuntu-artful/Dockerfile b/build/docker/ubuntu-artful/Dockerfile
new file mode 100644
index 0000000..1c8f963
--- /dev/null
+++ b/build/docker/ubuntu-artful/Dockerfile
@@ -0,0 +1,255 @@
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+#
+# Apache Thrift Docker build environment for Ubuntu Artful
+# Using all stock Ubuntu Artful packaging except for:
+# - cpp: stock boost 1.62 in artful has a nasty bug so we use stock boost 1.63
+# - d: does not come with Ubuntu so we're installing the latest
+# - d: deimos for libevent and openssl omitted - not compatible / build errors
+# - dart: dart repository doesn't work with apt 1.4 in artful
+# - dotnetcore, because netcore is for 1.0.0-preview and 2.0.0 is out
+# - haxe: see THRIFT-4352, but test/haxe cores during testing
+# and hxcpp 3.4.64 is not compatible with artful
+#
+
+FROM buildpack-deps:artful-scm
+MAINTAINER Apache Thrift <dev@thrift.apache.org>
+ENV DEBIAN_FRONTEND noninteractive
+
+RUN apt-get update && \
+ apt-get dist-upgrade -y && \
+ apt-get install -y --no-install-recommends \
+ apt \
+ apt-transport-https \
+ apt-utils \
+ curl \
+ dirmngr \
+ software-properties-common \
+ wget
+
+# csharp (mono) - if we ever want a later version
+# RUN echo "deb http://download.mono-project.com/repo/debian xenial main" | tee /etc/apt/sources.list.d/mono.list && \
+# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys A6A19B38D3D831EF
+
+# dotnet (core) 2.0.0 - project isn't ready for this yet:
+# RUN curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > /etc/apt/trusted.gpg.d/microsoft.gpg && \
+# echo "deb [arch=amd64] https://packages.microsoft.com/repos/microsoft-ubuntu-xenial-prod xenial main" > /etc/apt/sources.list.d/dotnetdev.list
+
+# node.js (this step runs apt-get update internally) - if we ever want a later version
+# note: node 8.5 introduced some issues with directory handling / jsdoc / something... using 7.x for now
+# RUN curl -sL https://deb.nodesource.com/setup_7.x | bash
+
+### install general dependencies
+RUN apt-get install -y --no-install-recommends \
+`# General dependencies` \
+ bash-completion \
+ bison \
+ build-essential \
+ clang \
+ cmake \
+ debhelper \
+ flex \
+ gdb \
+ llvm \
+ ninja-build \
+ pkg-config \
+ valgrind \
+ vim
+ENV PATH /usr/lib/llvm-3.8/bin:$PATH
+
+# boost-1.62 has a terrible bug in boost::test, see https://svn.boost.org/trac10/ticket/12507
+RUN apt-get install -y --no-install-recommends \
+`# C++ dependencies` \
+ libboost1.63-all-dev \
+ libevent-dev \
+ libssl-dev \
+ qt5-default \
+ qtbase5-dev \
+ qtbase5-dev-tools
+
+RUN apt-get install -y --no-install-recommends \
+`# csharp (mono) dependencies` \
+ mono-devel
+
+RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EBCF975E5BA24D5E && \
+ wget http://master.dl.sourceforge.net/project/d-apt/files/d-apt.list -O /etc/apt/sources.list.d/d-apt.list && \
+ wget -qO - https://dlang.org/d-keyring.gpg | apt-key add - && \
+ apt-get update && \
+ apt-get install -y --no-install-recommends \
+ `# D dependencies` \
+ dmd-bin \
+ libphobos2-dev \
+ dub \
+ dfmt \
+ dscanner \
+ libevent-dev \
+ libssl-dev \
+ xdg-utils
+# libevent deimos doesn't seem to work so not enabling it:
+# RUN mkdir -p /usr/include/dmd/druntime/import/deimos /usr/include/dmd/druntime/import/C && \
+# curl -sSL https://github.com/D-Programming-Deimos/libevent/archive/master.tar.gz| tar xz && \
+# mv libevent-master/deimos/* /usr/include/dmd/druntime/import/deimos/ && \
+# mv libevent-master/C/* /usr/include/dmd/druntime/import/C/ && \
+# rm -rf libevent-master
+# openssl deimos doesn't work with openssl-1.0.2 so not enabling it:
+# RUN curl -sSL https://github.com/D-Programming-Deimos/openssl/archive/master.tar.gz| tar xz && \
+# mv openssl-master/deimos/* /usr/include/dmd/druntime/import/deimos/ && \
+# mv openssl-master/C/* /usr/include/dmd/druntime/import/C/ && \
+# rm -rf openssl-master
+
+# dart is disabled because the repository won't sync properly with apt 1.4
+# RUN curl https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - && \
+# curl https://storage.googleapis.com/download.dartlang.org/linux/debian/dart_stable.list > /etc/apt/sources.list.d/dart_stable.list && \
+# apt-get update && \
+# apt-get install -y --no-install-recommends \
+# `# Dart dependencies` \
+# dart
+# ENV PATH /usr/lib/dart/bin:$PATH
+
+# project isn't ready for this quite yet:
+# RUN apt-get install -y --no-install-recommends \
+# `# dotnet core dependencies` \
+# dotnet-sdk-2.0.0
+
+RUN apt-get install -y --no-install-recommends \
+`# Erlang dependencies` \
+ erlang-base \
+ erlang-eunit \
+ erlang-dev \
+ erlang-tools \
+ rebar
+
+RUN apt-get install -y --no-install-recommends \
+`# GlibC dependencies` \
+ libglib2.0-dev
+
+RUN apt-get install -y --no-install-recommends \
+`# golang (go) dependencies` \
+ golang-go \
+ golang-race-detector-runtime
+
+RUN apt-get install -y --no-install-recommends \
+`# Haskell dependencies` \
+ ghc \
+ cabal-install
+
+# see THRIFT-4352, test/haxe cores on artful
+# RUN apt-get install -y --no-install-recommends \
+# `# Haxe dependencies` \
+# haxe \
+# neko \
+# neko-dev
+# RUN haxelib setup --always /usr/share/haxe/lib && \
+# haxelib install --always hxcpp
+
+RUN apt-get install -y --no-install-recommends \
+`# Java dependencies` \
+ ant \
+ ant-optional \
+ openjdk-8-jdk \
+ maven
+
+RUN apt-get install -y --no-install-recommends \
+`# Lua dependencies` \
+ lua5.3 \
+ lua5.3-dev && \
+ ln -s /usr/bin/lua5.3 /usr/bin/lua && \
+ ln -s /usr/bin/luac5.3 /usr/bin/luac
+# https://bugs.launchpad.net/ubuntu/+source/lua5.3/+bug/1707212
+# lua5.3 does not install alternatives!
+
+RUN apt-get install -y --no-install-recommends \
+`# Node.js dependencies` \
+ nodejs \
+ npm
+
+RUN apt-get install -y --no-install-recommends \
+`# OCaml dependencies` \
+ ocaml \
+ opam && \
+ opam init --yes && \
+ opam install --yes oasis
+
+RUN apt-get install -y --no-install-recommends \
+`# Perl dependencies` \
+ libbit-vector-perl \
+ libclass-accessor-class-perl \
+ libcrypt-ssleay-perl \
+ libio-socket-ssl-perl \
+ libnet-ssleay-perl
+
+RUN apt-get install -y --no-install-recommends \
+`# Php dependencies` \
+ php \
+ php-cli \
+ php-dev \
+ php-pear \
+ re2c \
+ phpunit
+
+RUN apt-get install -y --no-install-recommends \
+`# Python dependencies` \
+ python-all \
+ python-all-dbg \
+ python-all-dev \
+ python-ipaddress \
+ python-pip \
+ python-setuptools \
+ python-six \
+ python-tornado \
+ python-twisted \
+ python-wheel \
+ python-zope.interface && \
+ pip install --upgrade backports.ssl_match_hostname
+
+RUN apt-get install -y --no-install-recommends \
+`# Python3 dependencies` \
+ python3-all \
+ python3-all-dbg \
+ python3-all-dev \
+ python3-pip \
+ python3-setuptools \
+ python3-six \
+ python3-tornado \
+ python3-twisted \
+ python3-wheel \
+ python3-zope.interface
+
+RUN apt-get install -y --no-install-recommends \
+`# Ruby dependencies` \
+ ruby \
+ ruby-dev \
+ ruby-bundler
+RUN gem install bundler --no-ri --no-rdoc
+
+RUN apt-get install -y --no-install-recommends \
+`# Rust dependencies` \
+ cargo \
+ rustc
+
+RUN apt-get install -y --no-install-recommends \
+`# Static Code Analysis dependencies` \
+ cppcheck \
+ sloccount && \
+ pip install flake8
+
+# Clean up
+RUN rm -rf /var/cache/apt/* && \
+ rm -rf /var/lib/apt/lists/* && \
+ rm -rf /tmp/* && \
+ rm -rf /var/tmp/*
+
+ENV THRIFT_ROOT /thrift
+RUN mkdir -p $THRIFT_ROOT/src
+COPY Dockerfile $THRIFT_ROOT/
+WORKDIR $THRIFT_ROOT/src
diff --git a/build/docker/ubuntu-trusty/Dockerfile b/build/docker/ubuntu-trusty/Dockerfile
index 857384b..db2041a 100644
--- a/build/docker/ubuntu-trusty/Dockerfile
+++ b/build/docker/ubuntu-trusty/Dockerfile
@@ -10,62 +10,52 @@
# See the License for the specific language governing permissions and
# limitations under the License.
-# Apache Thrift Docker build environment for Ubuntu
#
-# Known missing client libraries:
-# - dotnetcore
+# Apache Thrift Docker build environment for Ubuntu Trusty
+# Using all stock Ubuntu Trusty packaging except for:
+# - d: does not come with Ubuntu so we're installing 2.070.0
+# - dart: does not come with Ubuntu so we're installing 1.20.1
+# - dotnetcore, disabled because netcore is for 1.0.0-preview and 2.0.0 is out
+# - haxe, disabled because the distro comes with 3.0.0 and it cores while installing
+# - node.js, disabled because it is at 0.10.0 in the distro which is too old (need 4+)
+# - ocaml, disabled because it fails to install properly
+#
FROM buildpack-deps:trusty-scm
MAINTAINER Apache Thrift <dev@thrift.apache.org>
-
ENV DEBIAN_FRONTEND noninteractive
-# Add apt sources
-# CMAKE
-RUN apt-get update && \
- apt-get install -y --no-install-recommends software-properties-common && \
- add-apt-repository -y ppa:george-edison55/cmake-3.x
-
-# Erlang
-RUN echo 'deb http://packages.erlang-solutions.com/debian trusty contrib' > /etc/apt/sources.list.d/erlang_solutions.list && \
- curl -sSL https://packages.erlang-solutions.com/debian/erlang_solutions.asc | apt-key add -
-
-# Dart
-RUN curl https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - && \
- curl https://storage.googleapis.com/download.dartlang.org/linux/debian/dart_stable.list > /etc/apt/sources.list.d/dart_stable.list && \
- sed -i /etc/apt/sources.list.d/dart_stable.list -e 's/https:/http:/g'
-
-# Consider using mirror nearby when building locally
-# TODO: Provide option via --build-arg=...
-# RUN sed -i /etc/apt/sources.list -e 's!http://archive.ubuntu.com/ubuntu/!http://your/mirror/!g'
+RUN apt-get update && \
+ apt-get dist-upgrade -y && \
+ apt-get install -y --no-install-recommends \
+ apt \
+ apt-transport-https \
+ apt-utils \
+ curl \
+ dirmngr \
+ software-properties-common \
+ wget
RUN apt-get update && apt-get install -y --no-install-recommends \
`# General dependencies` \
+ bash-completion \
bison \
build-essential \
clang \
cmake \
debhelper \
flex \
+ gdb \
+ llvm \
ninja-build \
pkg-config \
-`# Included in buildpack-deps` \
-`# autoconf` \
-`# automake` \
-`# g++` \
-`# git` \
-`# libtool` \
-`# make`
+ valgrind \
+ vim
+ENV PATH /usr/lib/llvm-3.8/bin:$PATH
RUN apt-get install -y --no-install-recommends \
`# C++ dependencies` \
-`# libevent and OpenSSL are needed by D too` \
- libboost-dev \
- libboost-filesystem-dev \
- libboost-program-options-dev \
- libboost-system-dev \
- libboost-test-dev \
- libboost-thread-dev \
+ libboost-all-dev \
libevent-dev \
libssl-dev \
qt5-default \
@@ -73,6 +63,70 @@
qtbase5-dev-tools
RUN apt-get install -y --no-install-recommends \
+`# csharp (mono) dependencies` \
+ mono-devel
+
+RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys EBCF975E5BA24D5E && \
+ wget http://master.dl.sourceforge.net/project/d-apt/files/d-apt.list -O /etc/apt/sources.list.d/d-apt.list && \
+ wget -qO - https://dlang.org/d-keyring.gpg | apt-key add - && \
+ apt-get update && \
+ apt-get install -y --no-install-recommends \
+`# D dependencies` \
+ dmd-bin=2.070.2-0 \
+ libphobos2-dev=2.070.2-0 \
+ dub \
+ dfmt \
+ dscanner \
+ xdg-utils
+# RUN mkdir -p /usr/include/dmd/druntime/import/deimos /usr/include/dmd/druntime/import/C && \
+# curl -sSL https://github.com/D-Programming-Deimos/libevent/archive/master.tar.gz| tar xz && \
+# mv libevent-master/deimos/* /usr/include/dmd/druntime/import/deimos/ && \
+# mv libevent-master/C/* /usr/include/dmd/druntime/import/C/ && \
+# rm -rf libevent-master
+# RUN curl -sSL https://github.com/D-Programming-Deimos/openssl/archive/master.tar.gz| tar xz && \
+# mv openssl-master/deimos/* /usr/include/dmd/druntime/import/deimos/ && \
+# mv openssl-master/C/* /usr/include/dmd/druntime/import/C/ && \
+# rm -rf openssl-master
+
+RUN curl https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - && \
+ curl https://storage.googleapis.com/download.dartlang.org/linux/debian/dart_stable.list > /etc/apt/sources.list.d/dart_stable.list && \
+ apt-get update && \
+ apt-get install -y --no-install-recommends \
+`# Dart dependencies` \
+ dart=1.20.1-1
+ENV PATH /usr/lib/dart/bin:$PATH
+
+RUN apt-get install -y --no-install-recommends \
+`# Erlang dependencies` \
+ erlang-base \
+ erlang-eunit \
+ erlang-dev \
+ erlang-tools \
+ rebar
+
+RUN apt-get install -y --no-install-recommends \
+`# GlibC dependencies` \
+ libglib2.0-dev
+
+RUN apt-get install -y --no-install-recommends \
+`# golang (go) dependencies` \
+ golang-go
+
+RUN apt-get install -y --no-install-recommends \
+`# Haskell dependencies` \
+ ghc \
+ cabal-install
+
+# disabled because it cores while installing
+# RUN apt-get install -y --no-install-recommends \
+# `# Haxe dependencies` \
+# haxe \
+# neko \
+# neko-dev && \
+# haxelib setup /usr/share/haxe/lib && \
+# haxelib install hxcpp 3.2.102
+
+RUN apt-get install -y --no-install-recommends \
`# Java dependencies` \
ant \
ant-optional \
@@ -80,28 +134,25 @@
maven
RUN apt-get install -y --no-install-recommends \
-`# Python dependencies` \
-`# TODO:` \
-`# Install twisted and zope.interface via pip. we need twisted at ./configure time, otherwise` \
-`# py.twisted tests are skipped.` \
- python-all \
- python-all-dbg \
- python-all-dev \
- python-pip \
- python-setuptools \
- python-twisted \
- python-zope.interface \
- python3-all \
- python3-all-dbg \
- python3-all-dev \
- python3-setuptools \
- python3-pip
+`# Lua dependencies` \
+ lua5.1 \
+ lua5.1-dev
+
+# disabled because it is too old
+# RUN apt-get install -y --no-install-recommends \
+# `# Node.js dependencies` \
+# nodejs \
+# npm
+
+# disabled because it fails to install properly
+# RUN apt-get install -y --no-install-recommends \
+# `# OCaml dependencies` \
+# ocaml \
+# opam && \
+# opam init --yes && \
+# opam install --yes oasis
RUN apt-get install -y --no-install-recommends \
-`# Ruby dependencies` \
- ruby \
- ruby-dev \
- ruby-bundler \
`# Perl dependencies` \
libbit-vector-perl \
libclass-accessor-class-perl \
@@ -112,119 +163,60 @@
RUN apt-get install -y --no-install-recommends \
`# Php dependencies` \
php5 \
- php5-dev \
php5-cli \
+ php5-dev \
php-pear \
re2c \
- phpunit \
-`# GlibC dependencies` \
- libglib2.0-dev
-
-RUN apt-get update && apt-get install -y --no-install-recommends \
-`# Erlang dependencies` \
- erlang-base \
- erlang-eunit \
- erlang-dev \
- erlang-tools \
- rebar
+ phpunit
RUN apt-get install -y --no-install-recommends \
-`# Haskell dependencies` \
- ghc \
- cabal-install \
-`# Haxe dependencies` \
- neko \
- neko-dev \
- libneko0
-
-# Newer release of nodejs
-RUN curl -sL https://deb.nodesource.com/setup_4.x | bash
-RUN apt-get install -y --no-install-recommends \
-`# Node.js dependencies` \
- nodejs
-
-# Add mono package repository url to get latest version of mono
-RUN echo "deb http://download.mono-project.com/repo/debian trusty main" | tee /etc/apt/sources.list.d/mono.list
-RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys A6A19B38D3D831EF
-RUN apt-get update && apt-get install -y --no-install-recommends \
-`# CSharp dependencies` \
- mono-devel
+`# Python dependencies` \
+ python-all \
+ python-all-dbg \
+ python-all-dev \
+ python-pip \
+ python-setuptools \
+ python-six \
+ python-twisted \
+ python-wheel \
+ python-zope.interface \
+ python3-all \
+ python3-all-dbg \
+ python3-all-dev \
+ python3-pip \
+ python3-setuptools \
+ python3-six \
+ python3-wheel \
+ python3-zope.interface && \
+ pip install -U ipaddress backports.ssl_match_hostname tornado && \
+ pip3 install -U backports.ssl_match_hostname tornado
+# installing tornado by pip/pip3 instead of debian package
+# if we install the debian package, the build fails in py2
RUN apt-get install -y --no-install-recommends \
-`# D dependencies` \
- xdg-utils \
-`# Dart dependencies` \
- dart \
-`# Lua dependencies` \
- lua5.2 \
- lua5.2-dev \
-`# MinGW dependencies` \
- mingw32 \
- mingw32-binutils \
- mingw32-runtime \
- nsis \
-`# Clean up` \
- && rm -rf /var/cache/apt/* && \
+`# Ruby dependencies` \
+ ruby \
+ ruby-dev \
+ ruby-bundler
+RUN gem install bundler --no-ri --no-rdoc
+
+RUN apt-get install -y --no-install-recommends \
+`# Rust dependencies` \
+ cargo \
+ rustc
+
+RUN apt-get install -y --no-install-recommends \
+`# Static Code Analysis dependencies` \
+ cppcheck \
+ sloccount && \
+ pip install flake8
+
+# Clean up
+RUN rm -rf /var/cache/apt/* && \
rm -rf /var/lib/apt/lists/* && \
rm -rf /tmp/* && \
rm -rf /var/tmp/*
-# Ruby
-RUN gem install bundler --no-ri --no-rdoc
-
-# Python optional dependencies
-RUN pip2 install -U ipaddress backports.ssl_match_hostname tornado
-RUN pip3 install -U backports.ssl_match_hostname tornado
-
-# Go
-RUN curl -sSL https://storage.googleapis.com/golang/go1.4.3.linux-amd64.tar.gz | tar -C /usr/local/ -xz
-ENV PATH /usr/local/go/bin:$PATH
-
-# Haxe
-RUN mkdir -p /usr/lib/haxe && \
- wget -O - https://github.com/HaxeFoundation/haxe/releases/download/3.2.1/haxe-3.2.1-linux64.tar.gz | \
- tar -C /usr/lib/haxe --strip-components=1 -xz && \
- ln -s /usr/lib/haxe/haxe /usr/bin/haxe && \
- ln -s /usr/lib/haxe/haxelib /usr/bin/haxelib && \
- mkdir -p /usr/lib/haxe/lib && \
- chmod -R 777 /usr/lib/haxe/lib && \
- haxelib setup --always /usr/lib/haxe/lib && \
- haxelib install --always hxcpp 3.4.64
-
-# Node.js
-# temporarily removed since this breaks the build (and is not needed to test C# code)
-# RUN curl -sSL https://www.npmjs.com/install.sh | sh
-
-# D
-RUN curl -sSL http://downloads.dlang.org/releases/2.x/2.070.0/dmd_2.070.0-0_amd64.deb -o /tmp/dmd_2.070.0-0_amd64.deb && \
- dpkg -i /tmp/dmd_2.070.0-0_amd64.deb && \
- rm /tmp/dmd_2.070.0-0_amd64.deb && \
- curl -sSL https://github.com/D-Programming-Deimos/openssl/archive/master.tar.gz| tar xz && \
- curl -sSL https://github.com/D-Programming-Deimos/libevent/archive/master.tar.gz| tar xz && \
- mkdir -p /usr/include/dmd/druntime/import/deimos /usr/include/dmd/druntime/import/C && \
- mv libevent-master/deimos/* openssl-master/deimos/* /usr/include/dmd/druntime/import/deimos/ && \
- mv libevent-master/C/* openssl-master/C/* /usr/include/dmd/druntime/import/C/ && \
- rm -rf libevent-master openssl-master && \
- echo 'gcc -Wl,--no-as-needed $*' > /usr/local/bin/gcc-dmd && \
- chmod 755 /usr/local/bin/gcc-dmd && \
- echo 'CC=/usr/local/bin/gcc-dmd' >> /etc/dmd.conf
-
-# Dart
-ENV PATH /usr/lib/dart/bin:$PATH
-
-# OCaml
-RUN echo 'deb http://ppa.launchpad.net/avsm/ppa/ubuntu trusty main' > /etc/apt/sources.list.d/avsm-official-ocaml.list && \
- gpg --keyserver keyserver.ubuntu.com --recv 61707B09 && \
- gpg --export --armor 61707B09 | apt-key add - && \
- apt-get update && \
- apt-get install -y ocaml opam && \
- opam init && \
- opam install oasis
-
-# Rust
-RUN curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain 1.17.0
-ENV PATH /root/.cargo/bin:$PATH
-
ENV THRIFT_ROOT /thrift
RUN mkdir -p $THRIFT_ROOT/src
COPY Dockerfile $THRIFT_ROOT/
diff --git a/build/docker/ubuntu-trusty/Dockerfile.orig b/build/docker/ubuntu-trusty/Dockerfile.orig
new file mode 100644
index 0000000..857384b
--- /dev/null
+++ b/build/docker/ubuntu-trusty/Dockerfile.orig
@@ -0,0 +1,231 @@
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+
+# Apache Thrift Docker build environment for Ubuntu
+#
+# Known missing client libraries:
+# - dotnetcore
+
+FROM buildpack-deps:trusty-scm
+MAINTAINER Apache Thrift <dev@thrift.apache.org>
+
+ENV DEBIAN_FRONTEND noninteractive
+
+# Add apt sources
+# CMAKE
+RUN apt-get update && \
+ apt-get install -y --no-install-recommends software-properties-common && \
+ add-apt-repository -y ppa:george-edison55/cmake-3.x
+
+# Erlang
+RUN echo 'deb http://packages.erlang-solutions.com/debian trusty contrib' > /etc/apt/sources.list.d/erlang_solutions.list && \
+ curl -sSL https://packages.erlang-solutions.com/debian/erlang_solutions.asc | apt-key add -
+
+# Dart
+RUN curl https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - && \
+ curl https://storage.googleapis.com/download.dartlang.org/linux/debian/dart_stable.list > /etc/apt/sources.list.d/dart_stable.list && \
+ sed -i /etc/apt/sources.list.d/dart_stable.list -e 's/https:/http:/g'
+
+# Consider using mirror nearby when building locally
+# TODO: Provide option via --build-arg=...
+# RUN sed -i /etc/apt/sources.list -e 's!http://archive.ubuntu.com/ubuntu/!http://your/mirror/!g'
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+`# General dependencies` \
+ bison \
+ build-essential \
+ clang \
+ cmake \
+ debhelper \
+ flex \
+ ninja-build \
+ pkg-config \
+`# Included in buildpack-deps` \
+`# autoconf` \
+`# automake` \
+`# g++` \
+`# git` \
+`# libtool` \
+`# make`
+
+RUN apt-get install -y --no-install-recommends \
+`# C++ dependencies` \
+`# libevent and OpenSSL are needed by D too` \
+ libboost-dev \
+ libboost-filesystem-dev \
+ libboost-program-options-dev \
+ libboost-system-dev \
+ libboost-test-dev \
+ libboost-thread-dev \
+ libevent-dev \
+ libssl-dev \
+ qt5-default \
+ qtbase5-dev \
+ qtbase5-dev-tools
+
+RUN apt-get install -y --no-install-recommends \
+`# Java dependencies` \
+ ant \
+ ant-optional \
+ openjdk-7-jdk \
+ maven
+
+RUN apt-get install -y --no-install-recommends \
+`# Python dependencies` \
+`# TODO:` \
+`# Install twisted and zope.interface via pip. we need twisted at ./configure time, otherwise` \
+`# py.twisted tests are skipped.` \
+ python-all \
+ python-all-dbg \
+ python-all-dev \
+ python-pip \
+ python-setuptools \
+ python-twisted \
+ python-zope.interface \
+ python3-all \
+ python3-all-dbg \
+ python3-all-dev \
+ python3-setuptools \
+ python3-pip
+
+RUN apt-get install -y --no-install-recommends \
+`# Ruby dependencies` \
+ ruby \
+ ruby-dev \
+ ruby-bundler \
+`# Perl dependencies` \
+ libbit-vector-perl \
+ libclass-accessor-class-perl \
+ libcrypt-ssleay-perl \
+ libio-socket-ssl-perl \
+ libnet-ssleay-perl
+
+RUN apt-get install -y --no-install-recommends \
+`# Php dependencies` \
+ php5 \
+ php5-dev \
+ php5-cli \
+ php-pear \
+ re2c \
+ phpunit \
+`# GlibC dependencies` \
+ libglib2.0-dev
+
+RUN apt-get update && apt-get install -y --no-install-recommends \
+`# Erlang dependencies` \
+ erlang-base \
+ erlang-eunit \
+ erlang-dev \
+ erlang-tools \
+ rebar
+
+RUN apt-get install -y --no-install-recommends \
+`# Haskell dependencies` \
+ ghc \
+ cabal-install \
+`# Haxe dependencies` \
+ neko \
+ neko-dev \
+ libneko0
+
+# Newer release of nodejs
+RUN curl -sL https://deb.nodesource.com/setup_4.x | bash
+RUN apt-get install -y --no-install-recommends \
+`# Node.js dependencies` \
+ nodejs
+
+# Add mono package repository url to get latest version of mono
+RUN echo "deb http://download.mono-project.com/repo/debian trusty main" | tee /etc/apt/sources.list.d/mono.list
+RUN apt-key adv --keyserver keyserver.ubuntu.com --recv-keys A6A19B38D3D831EF
+RUN apt-get update && apt-get install -y --no-install-recommends \
+`# CSharp dependencies` \
+ mono-devel
+
+RUN apt-get install -y --no-install-recommends \
+`# D dependencies` \
+ xdg-utils \
+`# Dart dependencies` \
+ dart \
+`# Lua dependencies` \
+ lua5.2 \
+ lua5.2-dev \
+`# MinGW dependencies` \
+ mingw32 \
+ mingw32-binutils \
+ mingw32-runtime \
+ nsis \
+`# Clean up` \
+ && rm -rf /var/cache/apt/* && \
+ rm -rf /var/lib/apt/lists/* && \
+ rm -rf /tmp/* && \
+ rm -rf /var/tmp/*
+
+# Ruby
+RUN gem install bundler --no-ri --no-rdoc
+
+# Python optional dependencies
+RUN pip2 install -U ipaddress backports.ssl_match_hostname tornado
+RUN pip3 install -U backports.ssl_match_hostname tornado
+
+# Go
+RUN curl -sSL https://storage.googleapis.com/golang/go1.4.3.linux-amd64.tar.gz | tar -C /usr/local/ -xz
+ENV PATH /usr/local/go/bin:$PATH
+
+# Haxe
+RUN mkdir -p /usr/lib/haxe && \
+ wget -O - https://github.com/HaxeFoundation/haxe/releases/download/3.2.1/haxe-3.2.1-linux64.tar.gz | \
+ tar -C /usr/lib/haxe --strip-components=1 -xz && \
+ ln -s /usr/lib/haxe/haxe /usr/bin/haxe && \
+ ln -s /usr/lib/haxe/haxelib /usr/bin/haxelib && \
+ mkdir -p /usr/lib/haxe/lib && \
+ chmod -R 777 /usr/lib/haxe/lib && \
+ haxelib setup --always /usr/lib/haxe/lib && \
+ haxelib install --always hxcpp 3.4.64
+
+# Node.js
+# temporarily removed since this breaks the build (and is not needed to test C# code)
+# RUN curl -sSL https://www.npmjs.com/install.sh | sh
+
+# D
+RUN curl -sSL http://downloads.dlang.org/releases/2.x/2.070.0/dmd_2.070.0-0_amd64.deb -o /tmp/dmd_2.070.0-0_amd64.deb && \
+ dpkg -i /tmp/dmd_2.070.0-0_amd64.deb && \
+ rm /tmp/dmd_2.070.0-0_amd64.deb && \
+ curl -sSL https://github.com/D-Programming-Deimos/openssl/archive/master.tar.gz| tar xz && \
+ curl -sSL https://github.com/D-Programming-Deimos/libevent/archive/master.tar.gz| tar xz && \
+ mkdir -p /usr/include/dmd/druntime/import/deimos /usr/include/dmd/druntime/import/C && \
+ mv libevent-master/deimos/* openssl-master/deimos/* /usr/include/dmd/druntime/import/deimos/ && \
+ mv libevent-master/C/* openssl-master/C/* /usr/include/dmd/druntime/import/C/ && \
+ rm -rf libevent-master openssl-master && \
+ echo 'gcc -Wl,--no-as-needed $*' > /usr/local/bin/gcc-dmd && \
+ chmod 755 /usr/local/bin/gcc-dmd && \
+ echo 'CC=/usr/local/bin/gcc-dmd' >> /etc/dmd.conf
+
+# Dart
+ENV PATH /usr/lib/dart/bin:$PATH
+
+# OCaml
+RUN echo 'deb http://ppa.launchpad.net/avsm/ppa/ubuntu trusty main' > /etc/apt/sources.list.d/avsm-official-ocaml.list && \
+ gpg --keyserver keyserver.ubuntu.com --recv 61707B09 && \
+ gpg --export --armor 61707B09 | apt-key add - && \
+ apt-get update && \
+ apt-get install -y ocaml opam && \
+ opam init && \
+ opam install oasis
+
+# Rust
+RUN curl https://sh.rustup.rs -sSf | sh -s -- -y --default-toolchain 1.17.0
+ENV PATH /root/.cargo/bin:$PATH
+
+ENV THRIFT_ROOT /thrift
+RUN mkdir -p $THRIFT_ROOT/src
+COPY Dockerfile $THRIFT_ROOT/
+WORKDIR $THRIFT_ROOT/src
diff --git a/build/docker/ubuntu-xenial/Dockerfile b/build/docker/ubuntu-xenial/Dockerfile
index 560cf87..255b6e8 100644
--- a/build/docker/ubuntu-xenial/Dockerfile
+++ b/build/docker/ubuntu-xenial/Dockerfile
@@ -10,32 +10,40 @@
# See the License for the specific language governing permissions and
# limitations under the License.
+#
# Apache Thrift Docker build environment for Ubuntu Xenial
+# Using all stock Ubuntu Xenial packaging except for:
+# - d: does not come with Ubuntu so we're installing 2.073.2 for coverage
+# - dart: does not come with Ubuntu so we're installing 1.22.1 for coverage
+#
+
#
# Known missing or disabled libraries:
# - d: deimos for libevent and openssl omitted - not compatible / build errors
FROM buildpack-deps:xenial-scm
MAINTAINER Apache Thrift <dev@thrift.apache.org>
-
ENV DEBIAN_FRONTEND noninteractive
### Add apt repos
-RUN apt-get update && apt-get install -y --no-install-recommends apt apt-transport-https curl wget apt-utils
+RUN apt-get update && apt-get dist-upgrade -y && apt-get install -y --no-install-recommends apt apt-transport-https curl wget apt-utils
# csharp (mono)
-RUN echo "deb http://download.mono-project.com/repo/debian xenial main" | tee /etc/apt/sources.list.d/mono.list && \
- apt-key adv --keyserver keyserver.ubuntu.com --recv-keys A6A19B38D3D831EF
+# RUN echo "deb http://download.mono-project.com/repo/debian xenial main" | tee /etc/apt/sources.list.d/mono.list && \
+# apt-key adv --keyserver keyserver.ubuntu.com --recv-keys A6A19B38D3D831EF
# D
RUN wget http://master.dl.sourceforge.net/project/d-apt/files/d-apt.list -O /etc/apt/sources.list.d/d-apt.list && \
- apt-get update && apt-get -y --allow-unauthenticated install --reinstall d-apt-keyring
+ wget -qO - https://dlang.org/d-keyring.gpg | apt-key add -
+ENV D_VERSION 2.073.2-0
# Dart
RUN curl https://dl-ssl.google.com/linux/linux_signing_key.pub | apt-key add - && \
curl https://storage.googleapis.com/download.dartlang.org/linux/debian/dart_stable.list > /etc/apt/sources.list.d/dart_stable.list && \
sed -i /etc/apt/sources.list.d/dart_stable.list -e 's/https:/http:/g'
+# since ubuntu-artful can't run dart, we'll run 1.240 on xenial for now
+ENV DART_VERSION 1.24.2-1
# dotnet (core) 2.0.0
RUN curl https://packages.microsoft.com/keys/microsoft.asc | gpg --dearmor > /etc/apt/trusted.gpg.d/microsoft.gpg && \
@@ -43,11 +51,11 @@
# node.js (this step runs apt-get update internally)
# note: node 8.5 introduced some issues with directory handling / jsdoc / something... using 7.x for now
-RUN curl -sL https://deb.nodesource.com/setup_7.x | bash
+# RUN curl -sL https://deb.nodesource.com/setup_7.x | bash
### install general dependencies
-RUN apt-get install -y --no-install-recommends \
+RUN apt-get update && apt-get install -y --no-install-recommends \
`# General dependencies` \
bash-completion \
bison \
@@ -84,19 +92,23 @@
`# csharp (mono) dependencies` \
mono-devel
-RUN apt-get install -y --no-install-recommends \
+RUN apt-get install -y --allow-unauthenticated --no-install-recommends \
`# D dependencies` \
- dmd-bin \
+ dmd-bin=$D_VERSION \
+ libphobos2-dev=$D_VERSION \
+ dub \
+ dfmt \
+ dscanner \
libevent-dev \
libssl-dev \
xdg-utils
-# libevent deimos doesn't seem to work so disabling it:
+# libevent deimos doesn't seem to work so not enabling it:
# RUN mkdir -p /usr/include/dmd/druntime/import/deimos /usr/include/dmd/druntime/import/C && \
# curl -sSL https://github.com/D-Programming-Deimos/libevent/archive/master.tar.gz| tar xz && \
# mv libevent-master/deimos/* /usr/include/dmd/druntime/import/deimos/ && \
# mv libevent-master/C/* /usr/include/dmd/druntime/import/C/ && \
# rm -rf libevent-master
-# openssl deimos doesn't work with openssl-1.0.2 so disabling it:
+# openssl deimos doesn't work with openssl-1.0.2 so not enabling it:
# RUN curl -sSL https://github.com/D-Programming-Deimos/openssl/archive/master.tar.gz| tar xz && \
# mv openssl-master/deimos/* /usr/include/dmd/druntime/import/deimos/ && \
# mv openssl-master/C/* /usr/include/dmd/druntime/import/C/ && \
@@ -104,7 +116,7 @@
RUN apt-get install -y --no-install-recommends \
`# Dart dependencies` \
- dart
+ dart=$DART_VERSION
ENV PATH /usr/lib/dart/bin:$PATH
RUN apt-get install -y --no-install-recommends \
@@ -159,7 +171,9 @@
RUN apt-get install -y --no-install-recommends \
`# Node.js dependencies` \
- nodejs
+ nodejs \
+ npm && \
+ ln -s /usr/bin/nodejs /usr/bin/node
RUN apt-get install -y --no-install-recommends \
`# OCaml dependencies` \
@@ -222,9 +236,6 @@
cargo \
rustc
-# Update anything else left hanging
-RUN apt-get dist-upgrade -y
-
# Clean up
RUN rm -rf /var/cache/apt/* && \
rm -rf /var/lib/apt/lists/* && \
diff --git a/build/docker/vars.sh b/build/docker/vars.sh
new file mode 100755
index 0000000..752e282
--- /dev/null
+++ b/build/docker/vars.sh
@@ -0,0 +1,23 @@
+#!/bin/bash
+#
+# Licensed to the Apache Software Foundation (ASF) under one
+# or more contributor license agreements. See the NOTICE file
+# distributed with this work for additional information
+# regarding copyright ownership. The ASF licenses this file
+# to you under the Apache License, Version 2.0 (the
+# "License"); you may not use this file except in compliance
+# with the License. You may obtain a copy of the License at
+#
+# http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing,
+# software distributed under the License is distributed on an
+# "AS IS" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY
+# KIND, either express or implied. See the License for the
+# specific language governing permissions and limitations
+# under the License.
+#
+
+[[ ! -z "$DOCKER_USER" ]] || export DOCKER_USER=$DEFAULT_DOCKER_USER
+[[ ! -z "$DOCKER_REPO" ]] || export DOCKER_REPO=$DEFAULT_DOCKER_REPO
+[[ ! -z "$DOCKER_TAG" ]] || export DOCKER_TAG=$DOCKER_USER/$DOCKER_REPO:$DISTRO