Publishing Kotlin Multiplatform Swift Packages Using Google Cloud Storage and Cloud Run
By Nate Ebel, Android developer
In this post, we’ll detail our solution for publishing, and consuming, Kotlin Multiplatform Swift Packages. Our solution leverages a custom Gradle plugin publishing XCFrameworks to Google Cloud Storage and a Google Cloud Run service to download Swift Package binaries requested by XCode. With this solution in place, we’ve been able to more efficiently serve multiple Kotlin Multiplatform libraries to our iOS application.
This is a part of an ongoing series on our usage of Kotlin Multiplatform at Premise:
- Part 1: Kotlin Multiplatform at Premise
- Part 2: Kotlin Multiplatform Project Structure for Integrating with Brownfield Applications
- Part 3: Building a CI Pipeline for Kotlin Multiplatform Mobile Using GitHub Actions
- Part 4: Publishing Kotlin Multiplatform Swift Packages Using Google Cloud Storage and Cloud Run — This Post
- Part 5: Generating BuildConfig Files for a Kotlin Multiplatform Library — Coming Soon
- Part 6: Optimizing Local Build Times for Kotlin Multiplatform Mobile Projects — Coming Soon
Premise and Kotlin Multiplatform Swift Packages
We’ve been using Kotlin Multiplatform in production since early 2021 in the form of our mobile-shared project.
During that time, we’ve consumed our shared code as a Swift Package within our iOS application. The integration of that Swift Package has gone through several iterations.
- v1: Use the multiplatform-swiftpackage plugin to build the Swift Package and store the XCFramework binary in GitHub
- v2: Use our own custom Gradle plugin to build the Swift Package and store the XCFramework binary in GitHub
These two solutions were very similar. Build the XCFramework. Generate the Package.swiftfile. Check both into git with the desired version tag.
These approaches worked fine for a while, but eventually we started to pay the price for our simple initial solution.
An XCFramework binary can be pretty large. Ours were in the ballpark of 100MB. So checking 2–3 of these into each commit (1 for each iOS architecture) started to really increase the overall download size of our git repo.
Why is this a problem?
Well, Swift Package Manager does a full clone of the git repository when it has to sync package dependencies or reset package caches. So, when our developers had to switch branches with difference library versions, or had to reset their Xcode caches, they had to download GBs worth of XCFramework binaries; thereby dramatically slowing down their Xcode sync.
Once we determined what was causing this slow down, we knew we needed a new approach. We decided to move to a remote url distribution model for our Kotlin Multiplatform Swift Packages. Rather than check the binary into source control, we would upload the binary to some remote location, acquire the url to that resource, and add that url, along with the checksum, to our Package.swift file.
This is conceptually pretty simple. However, there wasn’t an ideal out-of-the-box solution at the time. So, we built a solution ourselves which has been serving us well in managing the publication and integration of Swift Packages for multiple Kotlin Multiplatform projects.
The solution is broken down into three parts:
- Uploading XCFramework binaries to Google Cloud Storage
- Publishing the Package.swift file to GitHub
- Consuming the Swift Package from Xcode using Google Cloud Run
Uploading XCFramework Binaries to Google Cloud Storage
As part of our Kotlin Multiplatform project, we have a custom Gradle plugin named SwiftPackagePlugin that adds several key tasks to our build.
class SwiftPackagePlugin : Plugin<Project> {
override fun apply(target: Project) {
target.registerMoveXCFrameworkTask()
target.registerZipFileTask()
target.registerSwiftPackageTask()
target.registerLocalSwiftPackageTask()
}
}registerMoveXCFrameworkTask moves a build XCFramework from the build directory to our desired output directory location
/**
* Creates a moveXCFramework Gradle task to move files from the
* /build/swiftpackage directory to a /swiftpackage output directory
*/
private fun Project.registerMoveXCFrameworkTask() {
tasks.register("moveXCFramework", Copy::class.java) {
group = TASK_GROUP_NAME
description = "Moves XCFramework into swiftpackage output directory"
dependsOn("assemble${rootProject.name.capitalized()}XCFramework")
from("${this@registerMoveXCFrameworkTask.buildDir}/$OUTPUT_FILE_DIR_NAME")
into(File(OUTPUT_FILE_DIR_NAME))
}
}registerZipFileTask creates a zip containing the build XCFrameworks
/**
* Creates a zipXCFramework Gradle task
*/
private fun Project.registerZipFileTask() {
tasks.register("zipXCFramework", Zip::class.java) {
group = TASK_GROUP_NAME
description = "Creates a ZIP file for the XCFramework"
// dependsOn assembleMobilesharedXCFramework task
dependsOn("assemble${rootProject.name.capitalized()}XCFramework")
archiveFileName.set(project.getZipFileName())
destinationDirectory.set(File("${this@registerZipFileTask.buildDir}/$OUTPUT_FILE_DIR_NAME"))
from("${this@registerZipFileTask.buildDir}/XCFrameworks/release")
}
}registerLocalSwiftPackageTask constructs a Package.swift file useful for locally testing changes in our iOS project
/**
* Creates a createLocalSwiftPackage Gradle task that generates a
* Package.swift file needed for the generated XCFramework so it
* can be consumed as a local Swift Package
*/
private fun Project.registerLocalSwiftPackageTask() {
tasks.register("createLocalSwiftPackage") {
group = TASK_GROUP_NAME
description = "Creates a local Swift package to distribute the XCFramework"
dependsOn("zipXCFramework")
finalizedBy("moveXCFramework")
doLast {
val packageDotSwiftFile = createPackageDotSwiftFile(
outputDirName = OUTPUT_FILE_DIR_NAME,
filename = OUTPUT_PACKAGE_CONFIG_FILE_NAME
)
val packageConfiguration = SwiftPackageConfiguration.local(
packageName = rootProject.name,
zipFileName = project.getZipFileName()
)
packageDotSwiftFile.writePackageDotSwiftFile(packageConfiguration)
}
}
}registerSwiftPackageTask uploads a zipped set of XCFrameworks to Google Cloud Storage, builds a SwiftPackageConfigurationand then constructs a Package.swift file pointing to this remote file.
/**
* Generates the Package.swift file needed for the generated
* XCFramework so it can be consumed as a Swift Package
*/
private fun Project.registerSwiftPackageTask() {
tasks.register("createSwiftPackage") {
group = TASK_GROUP_NAME
description = "Creates the Swift package to distribute the XCFramework"
dependsOn("zipXCFramework")
doLast {
val packageDotSwiftFile = createPackageDotSwiftFile(
outputDirName = OUTPUT_FILE_DIR_NAME,
filename = OUTPUT_PACKAGE_CONFIG_FILE_NAME
)
uploadeToCloudStorage(
projectId = project.swiftPackageGCPProjectId,
bucketName = project.swiftPackageBucketName,
packageName = project.getZipFileName(),
filePath = "${project.buildDir}/$OUTPUT_FILE_DIR_NAME/${project.getZipFileName()}"
)
val packageConfiguration = SwiftPackageConfiguration.remote(
packageName = rootProject.name,
zipFileName = project.getZipFileName(),
zipChecksum = project.zipFileChecksum().trim(),
distributionUrl = project.swiftPackageDistributionURL
)
packageDotSwiftFile.writePackageDotSwiftFile(packageConfiguration)
}
}
}Uploading to Google Cloud Storage
The implementation of uploadToCloudStorage looks like the following. This relies on the com.google.cloud:google-cloud-storage:<version> dependency for interacting with Cloud Storage. By extension this also means the task must be run from an environment authenticated with Google Cloud. Locally, this generally means authentication via application-default credentials. In CI, we use the google-github-actions/auth GitHub Action to authenticate via a service account that has permissions to upload to our desired Cloud Storage bucket.
/**
* Uploads a specified XCFramework zip file to a specified Cloud Storage bucket
*
* @param projectId The ID of your GCP project
* @param bucketName The Cloud Storage bucket, within the specific project, to upload zip files to
* @param packageName The name of the XCFramework zip file
* @param filePath Path to the file to upload
*/
private fun uploadeToCloudStorage(projectId: String, bucketName: String, packageName: String, filePath: String) {
val storage = StorageOptions.newBuilder().setProjectId(projectId).build().service
val blobId: BlobId = BlobId.of(bucketName, packageName)
val blobInfo: BlobInfo = BlobInfo.newBuilder(blobId).build()
// Optional: set a generation-match precondition to avoid potential race
// conditions and data corruptions. The request returns a 412 error if the
// preconditions are not met.
val precondition = if (storage[bucketName, packageName] == null) {
// For a target object that does not yet exist, set the DoesNotExist precondition.
// This will cause the request to fail if the object is created before the request runs.
Storage.BlobWriteOption.doesNotExist()
} else {
// If the destination already exists in your bucket, instead set a generation-match
// precondition. This will cause the request to fail if the existing object's generation
// changes before the request runs.
Storage.BlobWriteOption.generationMatch(
storage[bucketName, packageName].generation
)
}
storage.createFrom(blobInfo, Paths.get(filePath), precondition)
}Building a Package.swift file
To build the desired Package.swift file we construct an instance of a SwiftPackageConfiguration which will hold the values we want to write out to disk
private data class SwiftPackageConfiguration private constructor(
private val packageName: String,
private val swiftToolsVersion: String,
private val platforms: String,
private val zipFileName: String,
private val zipChecksum: String,
private val isLocal: Boolean,
private val distributionUrl: String,
) {
val templateProperties = mapOf(
"toolsVersion" to swiftToolsVersion,
"name" to packageName,
"zipName" to zipFileName,
"platforms" to platforms,
"isLocal" to isLocal,
"checksum" to zipChecksum,
"url" to "$distributionUrl/$zipFileName"
)
companion object {
internal val templateFile =
SwiftPackagePlugin::class.java.getResource("/templates/$OUTPUT_PACKAGE_CONFIG_FILE_NAME.template")
fun local(
packageName: String,
swiftToolsVersion: String = DEFAULT_SWIFT_TOOLS_VERSION,
platforms: String = DEFAULT_IOS_PLATFORM_VERSION,
zipFileName: String
) = SwiftPackageConfiguration(packageName, swiftToolsVersion, platforms, zipFileName, "", true, "")
fun remote(
packageName: String,
swiftToolsVersion: String = DEFAULT_SWIFT_TOOLS_VERSION,
platforms: String = DEFAULT_IOS_PLATFORM_VERSION,
zipFileName: String,
zipChecksum: String,
distributionUrl: String,
) = SwiftPackageConfiguration(packageName, swiftToolsVersion, platforms, zipFileName, zipChecksum, false, distributionUrl)
}
}With this class, we then configure our instance as follows
val packageConfiguration = SwiftPackageConfiguration.remote(
packageName = rootProject.name,
zipFileName = project.getZipFileName(),
zipChecksum = project.zipFileChecksum().trim(),
distributionUrl = project.swiftPackageDistributionURL
)With an instance of this SwiftPackageConfiguration we then build our Package.swift file by using a template and subsituting in our desired configuration values.
// Writes the Package.swift file by
// first loading from a template file and
// second subtituting in the config values
private fun File.writePackageDotSwiftFile(packageConfiguration: SwiftPackageConfiguration) {
SimpleTemplateEngine()
.createTemplate(SwiftPackageConfiguration.templateFile)
.make(packageConfiguration.templateProperties)
.writeTo(writer())
}The base template file lives in /src/main/resources/templates/Package.swift.template
// swift-tools-version:$toolsVersion
import PackageDescription
let package = Package(
name: "$name",
platforms: [
$platforms
],
products: [
.library(
name: "$name",
targets: ["$name"]
),
],
targets: [
<% if (isLocal) print """ .binaryTarget(
name: "$name",
path: "./${zipName}"
),""" else print """ .binaryTarget(
name: "$name",
url: "$url",
checksum: "$checksum"
),""" %>
]
)When the configuration is applied to the template, it resulst in an output Package.swift file that looks like the following:
// swift-tools-version:5.8.0
import PackageDescription
let package = Package(
name: "mobileshared",
platforms: [
.iOS(.v14)
],
products: [
.library(
name: "mobileshared",
targets: ["mobileshared"]
),
],
targets: [
.binaryTarget(
name: "mobileshared",
url: "https://<cloud run url>/swiftpackage/mobileshared-<version>.zip",
checksum: "<XCFramework zip checksum>"
),
]
)Note the url value. This is a url pointing to a Google Cloud Run service which will be described below. The purpose of this service is to give us a known location to build up our artifact urls so we can add them to this Package.swift file.
Additional plugin code
To round out the plugin implementation, we have a handful of of other configuration values and helper functions/classes.
private const val DEFAULT_SWIFT_TOOLS_VERSION = "5.8.0"
private const val DEFAULT_IOS_PLATFORM_VERSION = ".iOS(.v14)"
private val Project.swiftPackageGCPProjectId
get() = System.getenv("SWIFT_PACKAGE_BUCKET_GCP_PROJECT_ID") ?: ""
private val Project.swiftPackageBucketName
get() = System.getenv("SWIFT_PACKAGE_BUCKET_NAME") ?: ""
private val Project.swiftPackageDistributionURL
get() = System.getenv("SWIFT_PACKAGE_DISTRIBUTION_URL") ?: ""
private fun Project.createPackageDotSwiftFile(outputDirName: String, filename: String): File {
return File("$buildDir/$outputDirName", filename).apply {
parentFile.mkdirs()
createNewFile()
}
}
private const val TASK_GROUP_NAME = "Premise Swift Package"
private const val OUTPUT_PACKAGE_CONFIG_FILE_NAME = "Package.swift"
private const val OUTPUT_FILE_DIR_NAME = "swiftpackage"
private fun Project.getZipFileName() = "${rootProject.name}-${rootProject.version}.zip"
internal fun Project.zipFileChecksum(): String {
val outputPath = "${buildDir}/$OUTPUT_FILE_DIR_NAME"
logger.info("checksum for path: $outputPath")
logger.info("checksum for file: ${getZipFileName()}")
return File(outputPath, getZipFileName())
.takeIf { it.exists() }
?.let { zipFile ->
ByteArrayOutputStream().use { os ->
project.exec {
workingDir = File(outputPath)
executable = "swift"
args = listOf("package", "compute-checksum", zipFile.name)
standardOutput = os
}
os.toString()
}
} ?: ""
}
Publishing the Package.swift File to GitHub
With our SwiftPackagePlugin applied to our project, we can run the createSwiftPacakge task from CI to build the binaries, upload them to Google Cloud Storage, and generate a Package.swift referencing that version.
However, we will need to publish that Package.swift file to GitHub so Xcode can reference it, and use it to eventually download the binaries.
To do this, we are using GitHub actions. The following, is a snippet from our release build that includes the neccessary job steps to build and publish the Swift Package. (this does not include the neccessary gcloud env setup)
# runs our Gradle tasks for building and uploading
- name: Create Swift Package
uses: gradle/gradle-build-action@v2.4.2
with:
arguments: createSwiftPackage
# copies Package.swift to desired location in repo so Xcode sees it
- name: Prepare iOS Artifacts
run: |
cp -r shared/build/swiftpackage/Package.swift .
# commits the updated Package.swift and tags it so Xcode can find it
- name: Tag Release
run: |
git add --all
git commit -m "[skip ci] Update Package.swift to version ${{ env.ARTIFACT_VERSION }}"
git tag -a ${{ env.ARTIFACT_VERSION }} -m "mobile-shared v${{ env.ARTIFACT_VERSION }}"
git push
git push origin ${{ env.ARTIFACT_VERSION }}Consuming the Swift Package from Xcode Using Google Cloud Run
From Xcode, we can now point our project at GitHub and ask it for a specific tagged version. Xcode and Swift Package Manager will then
- download the repo
- examine the
Package.swiftfile for the tagged commit - extract the binaryTarget url
- attempt to download the binary
When building the Package.swift file, we construct the url using the name of the shared artifact, the version of that artifact, and add those values to a known service url.
https://<cloud run service url>/swiftpackage/<artifact name>-<version>.zip
That service is our swift-package-distribution-service .
It’s a Google Cloud Run service that is permissioned to access our Cloud Storage bucket and stream the requested files back for download.
Implementing swift-package-distribution-service
How you would go about implementing your own version of this type of service is for you to decide. We built our service with Python using Goblet. Goblet is a framework designed to streamline the creation og Google Cloud Services using Python.
With Goblet, our entire service looks like the following:
import io
import os
from goblet import Goblet, goblet_entrypoint, Response
from flask import stream_with_context
from google.cloud import storage
import base64
CHUNK_SIZE = 8192
BUCKET_NAME = os.environ.get("CLOUD_STORAGE_BUCKET")
SWIFT_DISTRIBUTION_USERNAME = os.environ.get("SWIFT_DISTRIBUTION_USERNAME")
SWIFT_DISTRIBUTION_TOKEN = os.environ.get("SWIFT_DISTRIBUTION_TOKEN")
app = Goblet(function_name="swift-package-distribution-service", backend="cloudrun", routes_type="cloudrun")
goblet_entrypoint(app)
@app.route('/swiftpackage/{name}')
def home(name: str):
auth_header = get_auth_token()
if not is_token_valid(auth_header):
return Response("Invalid token", None, 401)
swiftpackage_filename = f'{name}'
storage_client = storage.Client()
bucket = storage_client.get_bucket(BUCKET_NAME)
if bucket is None or not bucket.exists():
missing_bucket_msg = f'Bucket {bucket} does not exist'
app.log.info(missing_bucket_msg)
return Response(missing_bucket_msg, None, 404)
else:
app.log.info(f'Located bucket {bucket}')
swiftpackage_blob = bucket.get_blob(swiftpackage_filename)
if swiftpackage_blob is None or not swiftpackage_blob.exists():
missing_blob_msg = f'Requested Swift Package {swiftpackage_filename} does not exist'
app.log.info(missing_blob_msg)
return Response(missing_blob_msg, None, 404)
else:
app.log.info(f'Located file {swiftpackage_filename} and started download')
swiftpackage_content = swiftpackage_blob.download_as_bytes()
app.log.info("Streaming response")
return stream_response(swiftpackage_content, swiftpackage_filename)
def stream_response(content: bytes, filename: str):
return stream_with_context(
read_bytes_chunks(content)), \
200, \
{
"Content-Type": "application/octet-stream",
"Content-Disposition": f'attachment; filename={filename}'
}
def read_bytes_chunks(content: bytes):
app.log.info("Reading chunks")
content_stream = io.BytesIO(content)
with content_stream:
while 1:
buffer = content_stream.read(CHUNK_SIZE)
if buffer:
app.log.info("chunk")
yield buffer
else:
app.log.info("no chunks left")
break
def get_auth_token() -> str:
# We'll skip this specific implementation
# We include a token taken from .netrc on developers' and CI machines
# If the token is not present, or incorrect, we will reject the request
# This is used due to Swift Package Manager not allowing us to auth via gcloud
def is_token_valid(auth_header: str) -> bool:
# We'll skip this specific implementation
def decode_auth_header(auth_header: str) -> str:
# We'll skip this specific implementationWith this service deployed, we can use its url when building our Package.swift file.
https://<cloud run service url>/swiftpackage/<artifact name>-<version>.zip
And now, when Xcode tries to download the file referenced in Package.swift it makes a request against this service, validates the auth token, and downloads the desired XCFramework binaries.
From there, our iOS build is off and running.
How Has This Solution Worked?
We’ve been using this solution for 6+ months now without issue.
By enabling us to move away from checking binaries into GitHub, it dramatically improved package sync time for our iOS devs that were regularly switching between branches requiring different versions of our mobile-shared project.
Because the plugin, and service, are general purpose, we were able to apply the same plugin and deployment model for our second Kotlin Multiplatform project in the org.
Because the plugin and service were both already in place, applying this approach to the second project was trivial; not requiring any real new development work beyond configuring the plugin in the new project.
What Are The Drawbacks To This Solution?
Perhaps the biggest drawback to this solution is that it requires cloud resources for storage and request handling. In an organization already using cloud resources this is likely not an issue. However, for open source projects or hobbyists, this could be a significant barrier to entry; both in terms of cost required to run these resources and the administration/security of cloud resources/organizations/etc.
On a similar note, another drawback is that it’s not an off-the-shelf solution. It required dev work leveraging knowledge of Swift Package Manager, Gradle plugins, and Google Cloud. This might not be something every time has access to and could be an additional barrier to entry. For a more, off-the-shelf option, I recommend looking into KMMBridge from TouchLab.
Finally, another, albiet smaller, limitation of the current implementation is the ordering of binary upload and Package.swift file creation. If uploading the XCFramework binary to Cloud Storage succeeds, but creation of the Package.swift file fails, it would result in a stranded artifact in our Cloud Storage bucket. In practice, this wouldn’t result in any significant cost unless it was happening repeatedly; and in 6+ months of using the current implementation I don’t believe we’ve ever encountered this issue.
Premise is constantly looking to hire top-tier engineering talent to help us improve our front-end, mobile offerings, data and cloud infrastructure. Come find out why we’re consistently named among the top places to work in Silicon Valley by visiting our Careers page.
