Repository navigation
History Rewriting to remove SGX Private Key #1394
Description
Instruction to sync to master after rewrite
If you do not have local changes, you can simply do git reset --hard [HASHTAG-OF-NEW-HEAD]
Please use the following commands
# Add upstream if you have not yet done so
git remote add upstream https://github.com/dmlc/tvm
git remote add prefilter https://github.com/tqchen/tvm
git fetch upstream
git fetch prefilter
# checkout a local branch for safety reason,
# you can always reset your master to the old head
git checkout -b backup
git checkout master
# Bring your changes on top of the most recent prefilter branch
git rebase prefilter/prefilter
# sync your local changes to master
git rebase --onto upstream/master e316f03d2d2e0c06019b6d026b9696b7d3f67b8d
# force push to your upstream master
git push --forceHere e316f03d2d2e0c06019b6d026b9696b7d3f67b8d is the hashtag of the head before fiilter. The last command will take the commits between ``e316f03``` and your head and apply it to the upstream/master's HEAD
Context
The history has been rewritten
- An old history branch is kept at branch prefilter, and we will delete it after 0.4
This is to followup issue on #1189
So far in the SGX demo, we introduced private keys into the commit history. Although the private key is only used for demo purposes and poses no security concern, it triggers false alarm of security scanning tools.
Currently, the private keys are shallowed removed. But it can still trigger the alarm due to the fact that the file is still in git history.
I have created https://gist.github.com/tqchen/ca5f1b898e27035621130d87aa9bebaf to deeply filter the history to remove the file. With an example branch here https://github.com/tqchen/tvm/commits/filter that gives the result of filter-branch.
Note that filter-brach causes divergence from the current tree, and in order to bring this change to master, we have to force rewrite the history and push to master -- note that commits contributions are preserved, but it indeed will cause troubles for our contributors.
This is a decision that can not be made lightly. So this is an RFC post, to hear opinions from the community on whether we should do this or not. Please express your opinions in this thread.
This thread will remain open for one week before we reach a decision
Pros and Cons
- Pros: avoid security check alarm in the future, although it is a false alarm.
- Cons: require contributors to sync up with the master via history rewriting.
Activity
tqchen commentedon Jul 6, 2018
@dmlc/tvm-team @mnuyens
alex-weaver commentedon Jul 8, 2018
Regardless of whether this is actually carried out or not, I think the most important thing is (if possible) add a check to the CI build to prevent security keys from ever being merged in the future. This would at least prevent needing to make this same decision again.
yzhliu commentedon Jul 9, 2018
Agree with @alex-weaver that we should add checker to CI.
Personally I support to remove the key file from history since this can confuse people who are sensitive to security and we may lose our potential users.
mnuyens commentedon Jul 9, 2018
This doesn't remove all cases, it also needs to be removed from the history of the python file.
I'm working on prepping a branch that has it removed as well.
tqchen commentedon Jul 9, 2018
@mnuyens can you comment which specific python file that we need to remove?
It would be great if you can help to update https://gist.github.com/tqchen/ca5f1b898e27035621130d87aa9bebaf to be the one that contains the correct form
Because the master branch is still evolving, we will need an automatic script to do the filtering, Thanks!
mnuyens commentedon Jul 10, 2018
We don't need to remove a whole file just some lines from apps/sgx/prepare_test_libs.py.
I'll test out changes to the command today and upload one that I think works along with a example branch that I've run the command on to confirm that it still keeps everything the way it should be.
mnuyens commentedon Jul 10, 2018
Here is my command that both removes the pem file and removes the necessary lines from apps/sgx/prepare_test_libs.py: https://gist.github.com/mnuyens/78deb8c9ad3df0070e0514ce62717659. I'll update with the new branch once I get that up.
mnuyens commentedon Jul 10, 2018
Here is the branch I've made a sample: https://github.com/mnuyens/tvm/tree/revised. I'm going confirm with the security team that this solves any problems and will update with confirmation once that's done.
tqchen commentedon Jul 16, 2018
@mnuyens can you confirm if your script works?
mnuyens commentedon Jul 20, 2018
Still working with to security to confirm sorry for the delay.
mnuyens commentedon Aug 3, 2018
Go ahead with it.
tqchen commentedon Aug 3, 2018
After deliberation and discussion with the team, we are going to filter the history to remove the trace. This is a reminder that we are going to do the filtering branch in 24 hours and please rebase your code to master after it gets updated @dmlc/tvm-team
tqchen commentedon Aug 4, 2018
The master branch has been filtered and put back into protection mode, please rebase your code against master
tqchen commentedon Aug 4, 2018
Please see the beginning of the post on how to sync back to the current master
tqchen commentedon Aug 8, 2018
close this PR since rewriting is done
tqchen commentedon Sep 3, 2018
As a note, the prefilter branch is now deleted from the current main repo, it will be kept in my fork https://github.com/tqchen/tvm for a while, instruction has been updated