Mohd Pozi, Muhammad Syafiq and Sato, Yukinori (2025) A data-augmented model routing framework for efficient LLM deployment in edge–cloud environments. Journal of Supercomputing, 81 (1573). pp. 1-21. ISSN 15730484
Preview |
PDF
- Published Version
Available under License Attribution 4.0 International (CC BY 4.0). Download (3MB) | Preview |
Abstract
Large language model (LLM)-based program generation tasks are hindered by high computational demands. These challenges, along with high deployment costs, often pose a barrier to practical applications. To address these, we propose a novel data-augmented multi-LLM model routing approach that classifies prompts based on whether they should be processed on a weak LLM engine or a strong LLM. Experimental results show up to 16 times better efficiency compared to the existing cascaded approaches, while preserving the inference accuracy. Thus, the proposed method optimally allocates prompts across multiple LLMs, reducing computational costs while maintaining inference accuracy.
| Item Type: | Article |
|---|---|
| Uncontrolled Keywords: | Large Language Model, Program Generation, Data Augmentation, Routing, Cascading, MEC, Cloud |
| Subjects: | Q Science > QA Mathematics > QA76 Computer software |
| Divisions: | School of Computing |
| Depositing User: | Mr. Izhar Hj. Hashim |
| Date Deposited: | 03 Nov 2026 09:27 |
| Last Modified: | 03 Nov 2026 09:27 |
| URI: | https://repo.uum.edu.my/id/eprint/34405 |
Actions (login required)
 |
View Item |