<p> Last night a user reported that fog.io's integration with our API was not passing the scrub flag. This started a conversation on GitHub about users who were able to see prior data on a virtual server that was newly created, but did not have the scrub flag passed. We wanted to address these concerns to make sure that we are being transparent and to make it clear that customer security remains paramount. At no time was customer data "leaked" between accounts. This would require that a user explicitly not scrub their volume after destroying their server; in this instance data would be recoverable and should be considered not sensitive. </p>
<p> This is an issue that we cleared up earlier in the year with scrubbing the drive. Taking everything into consideration, we made this the default behavior for all destroys. However as utilization of our cloud went up, we saw that scrubbing was starting to cause degradation in performance and caused many destroys to run for an extended period of time. </p>
<p> We then made the decision to update this behavior into a separately controllable and user-initiated action which was called “scrubbing”. This was made public in the control panel as a simple check box on the destroy menu and inside the API as scrub_data – a boolean parameter. Given some of the usage pattern we observed with customers during the on-boarding process, whereby many customers rapidly created and destroyed servers, we mistakenly assumed that this should be the default initial behavior. As a result, we switched the default mode away from scrubbing to improve performance, given that customers would have complete control over this action themselves. </p>
<p> The second mistake that we made was not notifying our customers that use the API. We should have sent an email to let each of them know of this change in default behavior; that way they could make any appropriate code changes necessary, as well as have enough notice to roll out those changes before the new default API behavior went into production. </p>
<p> We were wrong on both counts. We failed to deliver that message explicitly via email, and we should have taken more factors into account when determining the default behavior for a feature– specifically the multitude of customer concerns other than performance. </p>
<h2>Resolution</h2> <p> Our first and immediate update is to ensure that a clean system is provided during creates, regardless of what method was taken for initiating a destroy. Engineers are updating the code base right now to ensure that will be the default behavior, and we will provide another notice when that code is live. </p>
<p> The scrub feature will remain, allowing customers to take an extra level of precaution if they choose to scrub the data after the delete. </p>
<p> As we've grown, we have also seen a need to greatly improve our communication with our customers regarding updates, changes, and features. If anyone has any concerns or questions, we would love to hear from you. Please feel free to email me directly at Moisey@digitalocean.com. </p>
<p> </p>
<p>Thank you,<br /> Moisey<br /> Cofounder DigitalOcean</p>
Comments (0)
Sign in to post comments.