-
Notifications
You must be signed in to change notification settings - Fork 16
impl: verify cli signature #562
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from 1 commit
Commits
Show all changes
11 commits
Select commit
Hold shift + click to select a range
4001f7d
impl: support for downloading and verifying cli signatures
fioan89 94e74f5
fix: class cast exception
fioan89 345371f
impl: embed the pgp public key as a plugin resource
fioan89 c7af603
chore: fix UTs related to CLI downloading
fioan89 8f5e559
fix: download the correct CLI signature for Windows
fioan89 ea0e4ea
Merge branch 'main' into impl-verify-cli-signature
fioan89 5945b6c
chore: next version is 2.22.0
fioan89 06f0feb
impl: strict URL validation for the connection screen
fioan89 f8414f9
impl: strict URL validation for the URI handling
fioan89 8623572
fix: transform to url only after we checked the validation result
fioan89 8838a8e
chore: update UT expected result
fioan89 File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Next
Next commit
impl: support for downloading and verifying cli signatures
- Loading branch information
commit 4001f7d22f254df047fd9322fcdfc5262c58f08e
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
239 changes: 159 additions & 80 deletions
239
src/main/kotlin/com/coder/gateway/cli/CoderCLIManager.kt
Large diffs are not rendered by default.
Oops, something went wrong.
29 changes: 29 additions & 0 deletions
29
src/main/kotlin/com/coder/gateway/cli/downloader/CoderDownloadApi.kt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,29 @@ | ||
package com.coder.gateway.cli.downloader | ||
|
||
import okhttp3.ResponseBody | ||
import retrofit2.Response | ||
import retrofit2.http.GET | ||
import retrofit2.http.Header | ||
import retrofit2.http.HeaderMap | ||
import retrofit2.http.Streaming | ||
import retrofit2.http.Url | ||
|
||
/** | ||
* Retrofit API for downloading CLI | ||
*/ | ||
interface CoderDownloadApi { | ||
@GET | ||
@Streaming | ||
suspend fun downloadCli( | ||
@Url url: String, | ||
@Header("If-None-Match") eTag: String? = null, | ||
@HeaderMap headers: Map<String, String> = emptyMap(), | ||
@Header("Accept-Encoding") acceptEncoding: String = "gzip", | ||
): Response<ResponseBody> | ||
|
||
@GET | ||
suspend fun downloadSignature( | ||
@Url url: String, | ||
@HeaderMap headers: Map<String, String> = emptyMap() | ||
): Response<ResponseBody> | ||
} |
238 changes: 238 additions & 0 deletions
238
src/main/kotlin/com/coder/gateway/cli/downloader/CoderDownloadService.kt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,238 @@ | ||
package com.coder.gateway.cli.downloader | ||
|
||
import com.coder.gateway.cli.ex.ResponseException | ||
import com.coder.gateway.settings.CoderSettings | ||
import com.coder.gateway.util.OS | ||
import com.coder.gateway.util.SemVer | ||
import com.coder.gateway.util.getHeaders | ||
import com.coder.gateway.util.getOS | ||
import com.coder.gateway.util.sha1 | ||
import com.intellij.openapi.diagnostic.Logger | ||
import kotlinx.coroutines.Dispatchers | ||
import kotlinx.coroutines.withContext | ||
import okhttp3.ResponseBody | ||
import retrofit2.Response | ||
import java.io.FileInputStream | ||
import java.net.HttpURLConnection.HTTP_NOT_FOUND | ||
import java.net.HttpURLConnection.HTTP_NOT_MODIFIED | ||
import java.net.HttpURLConnection.HTTP_OK | ||
import java.net.URI | ||
import java.net.URL | ||
import java.nio.file.Files | ||
import java.nio.file.Path | ||
import java.nio.file.StandardCopyOption | ||
import java.nio.file.StandardOpenOption | ||
import java.util.zip.GZIPInputStream | ||
import kotlin.io.path.name | ||
import kotlin.io.path.notExists | ||
|
||
/** | ||
* Handles the download steps of Coder CLI | ||
*/ | ||
class CoderDownloadService( | ||
private val settings: CoderSettings, | ||
private val downloadApi: CoderDownloadApi, | ||
private val deploymentUrl: URL, | ||
forceDownloadToData: Boolean, | ||
) { | ||
private val remoteBinaryURL: URL = settings.binSource(deploymentUrl) | ||
private val cliFinalDst: Path = settings.binPath(deploymentUrl, forceDownloadToData) | ||
private val cliTempDst: Path = cliFinalDst.resolveSibling("${cliFinalDst.name}.tmp") | ||
|
||
suspend fun downloadCli(buildVersion: String, showTextProgress: ((t: String) -> Unit)? = null): DownloadResult { | ||
val eTag = calculateLocalETag() | ||
if (eTag != null) { | ||
logger.info("Found existing binary at $cliFinalDst; calculated hash as $eTag") | ||
} | ||
val response = downloadApi.downloadCli( | ||
url = remoteBinaryURL.toString(), | ||
eTag = eTag?.let { "\"$it\"" }, | ||
headers = getRequestHeaders() | ||
) | ||
|
||
return when (response.code()) { | ||
HTTP_OK -> { | ||
logger.info("Downloading binary to temporary $cliTempDst") | ||
response.saveToDisk(cliTempDst, showTextProgress, buildVersion)?.makeExecutable() | ||
DownloadResult.Downloaded(remoteBinaryURL, cliTempDst) | ||
} | ||
|
||
HTTP_NOT_MODIFIED -> { | ||
logger.info("Using cached binary at $cliFinalDst") | ||
showTextProgress?.invoke("Using cached binary") | ||
DownloadResult.Skipped | ||
} | ||
|
||
else -> { | ||
throw ResponseException( | ||
"Unexpected response from $remoteBinaryURL", | ||
response.code() | ||
) | ||
} | ||
} | ||
} | ||
|
||
/** | ||
* Renames the temporary binary file to its original destination name. | ||
* The implementation will override sibling file that has the original | ||
* destination name. | ||
*/ | ||
suspend fun commit(): Path { | ||
return withContext(Dispatchers.IO) { | ||
logger.info("Renaming binary from $cliTempDst to $cliFinalDst") | ||
Files.move(cliTempDst, cliFinalDst, StandardCopyOption.REPLACE_EXISTING) | ||
cliFinalDst.makeExecutable() | ||
cliFinalDst | ||
} | ||
} | ||
|
||
/** | ||
* Cleans up the temporary binary file if it exists. | ||
*/ | ||
suspend fun cleanup() { | ||
withContext(Dispatchers.IO) { | ||
runCatching { Files.deleteIfExists(cliTempDst) } | ||
.onFailure { ex -> | ||
logger.warn("Failed to delete temporary CLI file: $cliTempDst", ex) | ||
} | ||
} | ||
} | ||
|
||
private fun calculateLocalETag(): String? { | ||
return try { | ||
if (cliFinalDst.notExists()) { | ||
return null | ||
} | ||
sha1(FileInputStream(cliFinalDst.toFile())) | ||
} catch (e: Exception) { | ||
logger.warn("Unable to calculate hash for $cliFinalDst", e) | ||
null | ||
} | ||
} | ||
|
||
private fun getRequestHeaders(): Map<String, String> { | ||
return if (settings.headerCommand.isBlank()) { | ||
emptyMap() | ||
} else { | ||
getHeaders(deploymentUrl, settings.headerCommand) | ||
} | ||
} | ||
|
||
private fun Response<ResponseBody>.saveToDisk( | ||
localPath: Path, | ||
showTextProgress: ((t: String) -> Unit)? = null, | ||
buildVersion: String? = null | ||
): Path? { | ||
val responseBody = this.body() ?: return null | ||
Files.deleteIfExists(localPath) | ||
Files.createDirectories(localPath.parent) | ||
|
||
val outputStream = Files.newOutputStream( | ||
localPath, | ||
StandardOpenOption.CREATE, | ||
StandardOpenOption.TRUNCATE_EXISTING | ||
) | ||
val contentEncoding = this.headers()["Content-Encoding"] | ||
val sourceStream = if (contentEncoding?.contains("gzip", ignoreCase = true) == true) { | ||
GZIPInputStream(responseBody.byteStream()) | ||
} else { | ||
responseBody.byteStream() | ||
} | ||
|
||
val buffer = ByteArray(DEFAULT_BUFFER_SIZE) | ||
var bytesRead: Int | ||
var totalRead = 0L | ||
// local path is a temporary filename, reporting the progress with the real name | ||
val binaryName = localPath.name.removeSuffix(".tmp") | ||
sourceStream.use { source -> | ||
outputStream.use { sink -> | ||
while (source.read(buffer).also { bytesRead = it } != -1) { | ||
sink.write(buffer, 0, bytesRead) | ||
totalRead += bytesRead | ||
val prettyBuildVersion = buildVersion ?: "" | ||
showTextProgress?.invoke( | ||
"$binaryName $prettyBuildVersion - ${totalRead.toHumanReadableSize()} downloaded" | ||
) | ||
} | ||
} | ||
} | ||
return cliFinalDst | ||
} | ||
|
||
|
||
private fun Path.makeExecutable() { | ||
if (getOS() != OS.WINDOWS) { | ||
logger.info("Making $this executable...") | ||
this.toFile().setExecutable(true) | ||
} | ||
} | ||
|
||
private fun Long.toHumanReadableSize(): String { | ||
if (this < 1024) return "$this B" | ||
|
||
val kb = this / 1024.0 | ||
if (kb < 1024) return String.format("%.1f KB", kb) | ||
|
||
val mb = kb / 1024.0 | ||
if (mb < 1024) return String.format("%.1f MB", mb) | ||
|
||
val gb = mb / 1024.0 | ||
return String.format("%.1f GB", gb) | ||
} | ||
|
||
suspend fun downloadSignature(showTextProgress: ((t: String) -> Unit)? = null): DownloadResult { | ||
return downloadSignature(remoteBinaryURL, showTextProgress, getRequestHeaders()) | ||
} | ||
|
||
private suspend fun downloadSignature( | ||
url: URL, | ||
showTextProgress: ((t: String) -> Unit)? = null, | ||
headers: Map<String, String> = emptyMap() | ||
): DownloadResult { | ||
val signatureURL = url.toURI().resolve(settings.defaultSignatureNameByOsAndArch).toURL() | ||
val localSignaturePath = cliFinalDst.parent.resolve(settings.defaultSignatureNameByOsAndArch) | ||
logger.info("Downloading signature from $signatureURL") | ||
|
||
val response = downloadApi.downloadSignature( | ||
url = signatureURL.toString(), | ||
headers = headers | ||
) | ||
|
||
return when (response.code()) { | ||
HTTP_OK -> { | ||
response.saveToDisk(localSignaturePath, showTextProgress) | ||
DownloadResult.Downloaded(signatureURL, localSignaturePath) | ||
} | ||
|
||
HTTP_NOT_FOUND -> { | ||
logger.warn("Signature file not found at $signatureURL") | ||
DownloadResult.NotFound | ||
} | ||
|
||
else -> { | ||
DownloadResult.Failed( | ||
ResponseException( | ||
"Failed to download signature from $signatureURL", | ||
response.code() | ||
) | ||
) | ||
} | ||
} | ||
|
||
} | ||
|
||
suspend fun downloadReleasesSignature( | ||
buildVersion: String, | ||
showTextProgress: ((t: String) -> Unit)? = null | ||
): DownloadResult { | ||
val semVer = SemVer.parse(buildVersion) | ||
return downloadSignature( | ||
URI.create("https://releases.coder.com/coder-cli/${semVer.major}.${semVer.minor}.${semVer.patch}/").toURL(), | ||
showTextProgress | ||
) | ||
} | ||
|
||
companion object { | ||
val logger = Logger.getInstance(CoderDownloadService::class.java.simpleName) | ||
} | ||
} |
23 changes: 23 additions & 0 deletions
23
src/main/kotlin/com/coder/gateway/cli/downloader/DownloadResult.kt
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,23 @@ | ||
package com.coder.gateway.cli.downloader | ||
|
||
import java.net.URL | ||
import java.nio.file.Path | ||
|
||
|
||
/** | ||
* Result of a download operation | ||
*/ | ||
sealed class DownloadResult { | ||
object Skipped : DownloadResult() | ||
object NotFound : DownloadResult() | ||
data class Downloaded(val source: URL, val dst: Path) : DownloadResult() | ||
data class Failed(val error: Exception) : DownloadResult() | ||
|
||
fun isSkipped(): Boolean = this is Skipped | ||
|
||
fun isNotFound(): Boolean = this is NotFound | ||
|
||
fun isFailed(): Boolean = this is Failed | ||
|
||
fun isNotDownloaded(): Boolean = this !is Downloaded | ||
} |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.