In the process of creating images, some of Joyent’s internal-use SSH public keys were inadvertently left in certain published images. This led to the risk of potential unauthorized access to instances using the affected images.
Joyent acknowledges the assistance of an Open Source user in discovering this issue.
Joyent creates and publishes images to our Triton public cloud. These images are of various operating systems, to be used by customers in creating instances that run on the cloud.
We also make these images available to customers running Triton in their own datacenters, and to stand-alone SmartOS users.
An Open Source user reported to Joyent that certain images contained some
unexpected public key data. Joyent investigated and confirmed that – due to a
new bug in the image-building system – the following images were not properly
stripped of SSH public keys used during the development process and still
contain these keys in the
- ubuntu-certified-16.04-20190122 81c0ef69-e9d7-4e93-a15b-efd7ea9c9ee8
- ubuntu-certified-18.04-20190122 c9db249c-93ba-4507-9fa4-b4d0f81265fc
As part of the instance creation process, these keys would be propagated to the running instances that were using these images. These keys would make it possible for the Joyent developers (and Joyent development systems with the corresponding private key) to access these customer systems.
To the best of our knowledge, no such access was ever attempted in any systems using these images.
Actions Taken by Joyent
- The images listed above were disabled in our Triton public cloud, so that no further instances could be created from them.
- An audit was conducted to inventory all Triton public cloud instances created from the affected images, and all affected public cloud customers will be notified within hours of this advisory’s first posting.
- As a good-faith measure, the applicable SSH keys of Joyent employees/systems were all rotated out.
Actions You Need to Take
Triton Enterprise Software Users and Triton Public Cloud Users
The keys left in the image have the following fingerprints:
f9:41:4a:eb:84:c1:28:5e:31:27:c7:e8:c8:46:84:27 e2:2e:1d:40:d2:06:1d:2b:e1:27:3e:af:e8:f9:95:d9 35:35:c6:9a:f8:f3:d3:46:b7:4c:d9:9d:82:6b:3d:60 09:46:06:e8:9c:58:94:cb:8e:f2:7e:bf:a6:2b:a8:76 6f:7b:b7:f6:f4:1a:a1:28:70:fb:33:03:ec:04:fb:b5 83:8b:80:66:7a:24:d5:44:81:b3:eb:4a:eb:31:b5:b1 53:6e:35:43:1a:8a:b9:ec:36:1e:12:20:70:6b:69:76 57:4f:70:c6:bb:b6:2b:53:bc:71:aa:64:cd:30:5e:a7 7e:e0:00:29:fc:5e:5b:89:b3:03:71:12:0e:43:53:00 aa:f6:5e:c9:bf:9a:72:8c:cc:04:13:a1:e5:14:8f:3b
SHA256:scjD+2EYAJRz8wBreM9iXCk0oJ4NKTvTy2NgzlZG8JQ SHA256:0dR6/B605RKNpw8SZbAU6fcvTH7ylXQW079QeygaBDM SHA256:EMBapnoGdvBYoFLu912LHBG3eBoaGKMl8mq3wY5x5pI SHA256:tWhV06pPcS0A78tb5PBeWQ1cG27GUIDfmuX6/ze36tU SHA256:sxr9LU+d+8+y5YI6i6U8mGeT6/DKmEnI/g8SyC+XHs8 SHA256:+gcoJKBMgUbtSaA0xTSdzNY48Qr8zx0C93mjX7NV5Tw SHA256:XmQD/GojiNxYVATLspPwa2Zx5hdb4+iyFWHXcHhugIw SHA256:qaWrGQrhbYDQHkVK3J5AEHt3QEXlReWDgei4m29cz04 SHA256:Kp1Vo58n2uMup1NQwzQ8ympSTsiO2iFylpvPBiOVy5Y SHA256:XQ7XuhLmsCx04uthU8z+GhFZB4xiE+Lh+qsFzNNaOeY
If you have any non-deleted instance(s) created from the images listed in the “Description” part of the Overvew above (running in the Triton public cloud, or in your on-premises datacenter), proceed as follows:
- Check your instances to ensure that any user’s .ssh/authorized_keys file contains all of the keys for which you control the matching private key, and only those keys. If an instance’s file contains any other keys with fingerprints matching the above list, it may have been affected.
- Affected instances should be destroyed and recreated using unaffected images, to ensure that unauthorized Joyent SSH keys are not present. Note: Before destroying an instance, back up any data you wish to retain.
Open Source Triton Users
Please direct any further questions to The SmartOS Community Mailing Lists and IRC.
If you are a Joyent customer and have any further questions or concerns after reading the information provided above, please contact Joyent Support.
As noted above, if you are an Open Source Triton user, please direct any further questions to the SmartOS Community Mailing Lists and IRC.