Synthetix Response to Oracle Incident

Unfortunately this morning (3am Sydney time) an error occurred in our FX price feeds affecting the KRW price. We use a number of commercial API’s for price feeds across forex, commodities and cryptocurrencies. This morning one of these API’s began to intermittently report a price 1000x higher than the current rate for Korean Won. Our price oracle has a mechanism for discarding outliers and should have absorbed this discrepancy gracefully, unfortunately the price feed for KRW was only being served by two API’s at that time due to an earlier unrelated outage which had not been caught by our exception reporting. This meant the Oracle took an average of the only two remaining prices which caused it to misreport the price of KRW to our exchange rates contract. We have added additional redundancy to the price feeds and have added more robust exception handling to ensure that an error like this does not recur.

There are currently a number of trading bots actively trading on Synthetix.exchange using different strategies, one of these bots was able to detect this price error and exploit it to trade into and out of sKRW during the window where the API was incorrectly reporting the price. This resulted in several trades with profits of 1000x, resulting in over $1b in profit in less than an hour.

We were alerted by our community and our CTO stopped the Oracle service from sending rates to the contracts, this had the effect of halting all transfers and trading within the system.

We have since been in contact with the owner of the bot, who has agreed to reverse the trades in exchange for a bug bounty.  

We have now resumed trading and transfers within the system.

We are very conscious of the challenges in using oracles for price feeds and we will continue to improve this and all other aspects of the Synthetix system. Chainlink is a key partner in our move towards a more robust and decentralised oracle, and we look forward to working closely with their team to improve our Oracle.

Thank you to our community of SNX stakers and Synth traders for their patience during this incident. We will continue to improve Synthetix.Exchange and build the best trading experience possible.