Gradle Build System in Apache Beam Overview Apache Beam is a mono-repo using Gradle as its build system. The entire project (Java, Python, Go, website) is managed as a single Gradle project. Key Files build.gradle.kts - Root build configuration settings.gradle.kts - Project structure and module definitions gradle.properties - Global properties and versions buildSrc/ - Custom Gradle plugins including BeamModulePlugin BeamModulePlugin Located at buildSrc/src/main/groovy/org/apache/beam/gradle/BeamModulePlugin.groovy Purpose Manages Java dependencies centrally Configures project types (Java, Python, Go, Proto, Docker, etc.) Defines common custom tasks Java Project Configuration apply plugin : 'org.apache.beam.module' applyJavaNature ( automaticModuleName : 'org.apache.beam.sdk.io.kafka' ) Common Commands Build

Build entire project

./gradlew build

Build specific project

./gradlew :sdks:java:core:build ./gradlew -p sdks/java/core build

Compile only (no tests)

./gradlew :sdks:java:core:compileJava Test

Run tests

./gradlew :sdks:java:core:test

Run specific test

./gradlew :sdks:java:core:test --tests *MyTest

Skip tests

./gradlew build -x test Clean

Clean specific project

./gradlew :sdks:java:core:clean

Clean everything

./gradlew clean Formatting

Java formatting (Spotless)

./gradlew spotlessApply

Check formatting

./gradlew spotlessCheck

Format CHANGES.md

./gradlew formatChanges Publishing

Publish to Maven Local

./gradlew -Ppublishing :sdks:java:core:publishToMavenLocal

Publish all Java artifacts

./gradlew -Ppublishing publishToMavenLocal Pre-commit Tasks Java ./gradlew javaPreCommit Python ./gradlew pythonPreCommit Combined ./gradlew :checkSetup

Validates Go, Java, Python environments

Useful Flags Flag Description -p Run task in specific project directory -x Exclude task --tests Filter tests -Ppublishing Enable publishing tasks -PdisableSpotlessCheck=true Disable formatting check -PdisableCheckStyle=true Disable checkstyle -PskipCheckerFramework Skip Checker Framework --continue Continue after failures --info Verbose output --debug Debug output --scan Generate build scan --parallel Parallel execution GCP-related Properties -PgcpProject = my-project -PgcpRegion = us-central1 -PgcpTempRoot = gs://bucket/temp -PgcsTempRoot = gs://bucket/temp Docker Tasks

Build Java SDK container

./gradlew :sdks:java:container:java11:docker

Build Python SDK container

./gradlew :sdks:python:container:py312:docker

Build and push a container into a custom repository

./gradlew :sdks:java:container:java11:docker \ -Pdocker-repository-root = gcr.io/project \ -Pdocker-tag = custom \ -Ppush-containers If a :docker task produces logs that contain the following: WARNING: No output specified with docker-container driver. Build result will only remain in the build cache. then you must use -PuseDockerBuildx when running :docker tasks in this environment. For example:

Build and push a go container into a custom repository

./gradlew :sdks:go:container:docker \ -Pdocker-repository-root=gcr.io/project \ -Pdocker-tag=custom \ -Ppush-containers \ -PuseDockerBuildx Dependency Management View Dependencies ./gradlew :sdks:java:core:dependencies ./gradlew :sdks:java:core:dependencies --configuration runtimeClasspath Force Dependency Version In build.gradle : configurations . all { resolutionStrategy . force 'com.google.guava:guava:32.0.0-jre' } Troubleshooting Clean Gradle Cache rm -rf ~/.gradle/caches rm -rf .gradle rm -rf build Common Errors NoClassDefFoundError Run ./gradlew clean Delete gradle cache Proto-related Errors Regenerate protos: ./gradlew generateProtos Dependency Conflicts Check dependencies: ./gradlew dependencies Use --scan for detailed analysis Useful Tasks

List all tasks

./gradlew tasks

List tasks for a project

./gradlew :sdks:java:core:tasks

Show project structure

./gradlew projects IDE Integration IntelliJ Open repository root as Gradle project Wait for indexing Gradle tool window shows all tasks VS Code Install Gradle extension for task discovery