AvalancheGo v1.9.12 X-Chain Regression: Insights and Analysis
The Avalanche Primary Network encountered brief periods of instability, lasting 134 minutes on 3/22/23 and 82 minutes on 3/26/23. This was due to a regression in OperationTx verification introduced during a refactor of the X-Chain. Avalanche Subnets, operating independently, continued to finalize blocks unaffected during these periods.
Issue: Erroneous X-Chain Refactor to Prepare for Cortina (036e34)
- The execution of X-Chain state transitions was consolidated into a single location as part of preparation for Cortina.
- However, the handling of additional OperationTx UTXOs was not properly migrated, leading to discrepancies.
- This refactor was introduced in AvalancheGo v1.9.12.
To address this issue, the missing logic from v1.9.11 OperationTx handling was reintroduced in v1.9.14, ensuring proper transaction execution.
Impact: Network Instability and Fixes
During the transition from v1.9.12 to v1.9.14 and subsequently to v1.9.16:
- Block acceptance delays were observed on the C-Chain and P-Chain.
- Validators running different versions contributed to the instability until the majority upgraded to v1.9.15 and v1.9.16.
- Subnets continued to function normally, maintaining block finalization.
Looking Forward
Steps taken and planned for resolution and improvement:
- Regression testing on the X-Chain will be intensified before Cortina activation.
- Enhancements in X-Chain state merklization and Subnet UX improvements are ongoing.
We appreciate the community’s collaboration in identifying issues and their swift response in upgrading nodes, contributing to network stability.
https://medium.com/avalancheavax/avalanchego-v1-9-12-x-chain-regression-9c89c1b3b1ac