{"@context":"https://schema.org","@type":"CreativeWork","@id":"https://forgecascade.org/public/capsules/bcf07308-1ed3-4539-90e8-cfcc72e578fb","identifier":"bcf07308-1ed3-4539-90e8-cfcc72e578fb","url":"https://forgecascade.org/public/capsules/bcf07308-1ed3-4539-90e8-cfcc72e578fb","name":"Carbon-Taxed Transformers: A Green Compression Pipeline for Overgrown Language Models","text":"# Carbon-Taxed Transformers: A Green Compression Pipeline for Overgrown Language Models\n\n**Authors:** Ajmain Inqiad Alam, Palash Roy, Chanchal K. Roy, Banani Roy, Kevin A. Schneider\n**arXiv:** https://arxiv.org/abs/2604.25903v1\n**Published:** 2026-04-28T17:48:16Z\n\n## Abstract\nThe accelerating adoption of Large Language Models (LLMs) in software engineering (SE) has brought with it a silent crisis: unsustainable computational cost. While these models demonstrate remarkable capabilities in different SE tasks, they are unmanageably large, slow to deploy, memory-intensive, and carbon-heavy. This reality threatens not only the scalability and accessibility of AI-powered SE, but also its long-term environmental sustainability. The research challenge is clear: we must go beyond accuracy and address efficiency and environmental cost as first-class design constraints. To meet this challenge, we introduce Carbon-Taxed Transformers (CTT), a systematic multi-architectural compression principled pipeline ordering inspired by economic carbon taxation principles. Drawing from the economic concept of carbon pricing, CTT operationalizes a computational carbon tax that penalizes architectural inefficiencies and rewards deployment-ready compression. We evaluate CTT across three core SE tasks: code clone detection, code summarization, and code generation, with models spanning encoder-only, encoder-decoder, and decoder-only architecture. Our results show that CTT delivers on inference: (1) up to 49x memory reduction, (2) time reduction up to 8-10x for clone detection, up to 3x for summarization, and 4-7x for generation, (3) up to 81% reduction in CO2 emissions and (4) CTT retains around 98% accuracy on clone detection, around 89% on summarization, and up to 91% (textual metrics) and 68% (pass@1) for generation. Two ablation studies show that pipeline ordering and individual component contributions are both essential, providing empirical justification for CTT's design and effectiveness. ","keywords":["cs.SE","cs.LG"],"about":[],"citation":[],"isPartOf":{"@type":"Dataset","name":"Forge Cascade Knowledge Graph","url":"https://forgecascade.org"},"publisher":{"@type":"Organization","name":"Forge Cascade","url":"https://forgecascade.org"},"dateCreated":"2026-04-29T06:00:03.529000Z","dateModified":"2026-05-08T23:41:26.874252Z","isBasedOn":"https://arxiv.org/abs/2604.25903v1","additionalProperty":[{"@type":"PropertyValue","name":"trust_level","value":65},{"@type":"PropertyValue","name":"verification_status","value":"source_linked"},{"@type":"PropertyValue","name":"provenance_status","value":"valid"},{"@type":"PropertyValue","name":"evidence_level","value":"primary_source"},{"@type":"PropertyValue","name":"content_hash","value":"71e2dbf389e73ac9ccd4373936a6ce8780bcdfe72e782146d5b8a1a6281a0ec4"}]}