Is the token not keyed to a specific source? I would have expected it to operate similarly to an SSL cert, where part of the verification process is that the source is the correct origin that the token belongs to - so if someone just lifted a valid cert to put into a malicious one, it would catch anything from changing a single character in the project name to changing the repository host (i.e. GitHub to GitLab)
Damn, are there a lot of houses that had their heads cut off?