LlaMario: controllable Mario level generation based on large language models

doi:10.11996/JG.j.2095-302X.2026030511

Abstract

Abstract:

In game development, Procedural Content Generation (PCG) effectively reduces development costs and enhances the diversity of level design. Recent advances in integrating PCG with machine learning have demonstrated significant progress in game level generation. However, existing methods still exhibit limitations in generation controllability and adaptability to complex design intentions. To address these limitations, the LlaMario model was proposed to convert Mario game levels into character matrices for training with Large Language Models (LLMs), and then transformed the generated symbolic sequences into playable Mario levels. Specifically, we adopted the Llama 3.1-8B-instruct model combined with the efficient language model to fine-tune the Unsloth framework, empowering the model with enhanced level comprehension and generation capabilities. An instruction set of 40 000 entries derived from the Gemini model’s comprehension of level data was constructed and employed together with the data-efficient Alpaca instruction-tuning strategy for dataset construction; the resulting LlaMario trained on this dataset generated highly playable Mario levels from natural language descriptions. Experimental results validated that the proposed model achieved exceptional performance in level logical coherence, playability, complexity, and generation quality, successfully producing user-intended Mario game levels. Furthermore, the proposed framework demonstrated generality and was extended to other tile-based level generation tasks.

Key words: procedural content generation, large language models, game level generation, instruction fine-tuning, controllable level generation

CLC Number:

TP317
TP18

GENG Yuxuan, LU Yinan, WU Tieru, LI Wenhui, MA Rui. LlaMario: controllable Mario level generation based on large language models[J]. Journal of Graphics, 2026, 47(3): 511-523.

Figures/Tables 14

References 27

[1]	SHAKER N, TOGELIUS J, NELSON M J. Procedural content generation in games[M]. Cham: Springer, 2016: 1-2.
[2]	RISI S, TOGELIUS J. Increasing generality in machine learning through procedural content generation[J]. Nature Machine Intelligence, 2020, 2(8): 428-436. DOI
[3]	GOODFELLOW I, POUGET-ABADIE J, MIRZA M, et al. Generative adversarial networks[J]. Communications of the ACM, 2020, 63(11): 139-144. DOI URL
[4]	GIACOMELLO E, LANZI P L, LOIACONO D. DOOM level generation using generative adversarial networks[C]//2018 IEEE Games, Entertainment, Media Conference. New York: IEEE Press, 2018: 316-323.
[5]	FONTAINE M C, LIU R L, KHALIFA A, et al. Illuminating mario scenes in the latent space of a generative adversarial network[EB/OL]. [2025-06-15]. https://aaai.org/papers/05922-illuminating-mario-scenes-in-the-latent-space-of-a-generative-adversarial-network/.
[6]	SUMMERVILLE A, SNODGRASS S, GUZDIAL M, et al. Procedural content generation via machine learning (PCGML)[J]. IEEE Transactions on Games, 2018, 10(3): 257-270. DOI URL
[7]	SUDHAKARAN S, GONZÁLEZ-DUQUE M, FREIBERGER M, et al. MarioGPT: open-ended text2level generation through large language models[C]//The 37th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2023: 2361.
[8]	SUMMERVILLE A J, SNODGRASS S, MATEAS M, et al. The VGLC: the video game level corpus[EB/OL]. [2025-06-15]. https://arxiv.org/abs/1606.07487.
[9]	TOUVRON H, LAVRIL T, IZACARD G, et al. LLaMA: open and efficient foundation language models[EB/OL]. [2025-06-15]. https://arxiv.org/abs/2302.13971.
[10]	RADFORD A, WU J, CHILD R, et al. Language models are unsupervised multitask learners[R]. San Francisco: OpenAI, 2019.
[11]	HAN D, HAN M, Unsloth Team. Unsloth[EB/OL]. [2025-02-21]. http://github.com/unslothai/unsloth.
[12]	BROWNE C, MAIRE F. Evolutionary game design[J]. IEEE Transactions on Computational Intelligence and AI in Games, 2010, 2(1): 1-16.
[13]	SMITH A M, MATEAS M. Answer set programming for procedural content generation: a design space approach[J]. IEEE Transactions on Computational Intelligence and AI in Games, 2011, 3(3): 187-200. DOI URL
[14]	LIU J L, SNODGRASS S, KHALIFA A, et al. Deep learning for procedural content generation[J]. Neural Computing and Applications, 2021, 33(1): 19-37. DOI
[15]	SUMMERVILLE A, MATEAS M. Super mario as a string: platformer level generation via LSTMs[EB/OL]. [2025-06-15]. https://doi.org/10.26503/dl.v2016i1.752.
[16]	BEUKMAN M, CLEGHORN C W, JAMES S. Procedural content generation using neuroevolution and novelty search for diverse video game levels[C]//The Genetic and Evolutionary Computation Conference. New York: ACM, 2022: 1028-1037.
[17]	VOLZ V, SCHRUM J, LIU J L, et al. Evolving mario levels in the latent space of a deep convolutional generative adversarial network[C]//The Genetic and Evolutionary Computation Conference. New York: ACM, 2018: 221-228.
[18]	GONZÁLEZ-DUQUE M, PALM R B, HAUBERG S, et al. Mario plays on a manifold: generating functional content in latent space through differential geometry[C]//2022 IEEE Conference on Games. New York: IEEE Press, 2022: 385-392.
[19]	SARKAR A, COOPER S. Generating and blending game levels via quality-diversity in the latent space of a variational autoencoder[C]//The 16th International Conference on the Foundations of Digital Games. New York: ACM, 2021: 3.
[20]	MOURET J B, CLUNE J. Illuminating search spaces by mapping elites[EB/OL]. [2025-06-15]. https://arxiv.org/abs/1504.04909.
[21]	TODD G, EARLE S, NASIR M U, et al. Level generation through large language models[C]//The 18th International Conference on the Foundations of Digital Games. New York: ACM, 2023: 70.
[22]	Gemini Team Google. Gemini: a family of highly capable multimodal models[EB/OL]. [2025-06-15]. https://arxiv.org/abs/2312.11805.
[23]	TAORI R, GULRAJANI I, ZHANG T Y, et al. Alpaca: a strong, replicable instruction-following model[R]. Stanford: Stanford Center for Research on Foundation Models, 2023.
[24]	SHINN N, CASSANO F, GOPINATH A, et al. Reflexion: language agents with verbal reinforcement learning[C]//The 37th International Conference on Neural Information Processing Systems. Red Hook: Curran Associates Inc., 2023: 377.
[25]	Amidos2006. Mario-AI-Framework[EB/OL]. [2025-02-21]. https://github.com/amidos2006/Mario-AI-Framework.
[26]	TOGELIUS J, KARAKOVSKIY S, BAUMGARTEN R. The 2009 Mario AI competition[C]//IEEE Congress on Evolutionary Computation. New York: IEEE Press, 2010: 1-8.
[27]	LI Y H, WANG H R, ZHANG C. Assessing logical puzzle solving in large language models: insights from a minesweeper case study[EB/OL]. [2025-06-15]. https://aclanthology.org/2024.naacl-long.4/.

瓷砖类型	字符表示	场景表示
空气	‘-’
不可破坏障碍物	‘X’
可破坏障碍物	‘S’
问号砖块	‘?’/‘Q’
硬币	‘o’
敌人	‘E’
左管道头	‘<’
右管道头	‘>’
左管道体	‘[’
右管道体	‘]’
炮台头	‘B’
炮台体	‘b’

瓷砖类型	字符表示	场景表示
空气	‘-’
不可破坏障碍物	‘X’
可破坏障碍物	‘S’
问号砖块	‘?’/‘Q’
硬币	‘o’
敌人	‘E’
左管道头	‘<’
右管道头	‘>’
左管道体	‘[’
右管道体	‘]’
炮台头	‘B’
炮台体	‘b’

N值	NRR↓	NRR-cs↓	ΔNRR↓	ΔNRR-cs↓
3	0.999 999	0.999 910	0.000 002	0.000 042
4	0.999 997	0.999 875	0.000 012	0.000 236
5	0.999 995	0.999 841	0.000 048	0.000 680

N值	NRR↓	NRR-cs↓	ΔNRR↓	ΔNRR-cs↓
3	0.999 999	0.999 910	0.000 002	0.000 042
4	0.999 997	0.999 875	0.000 012	0.000 236
5	0.999 995	0.999 841	0.000 048	0.000 680

关卡生成模型	关卡可玩性/%↑	A*与对应模型人工筛查差值
Llama 3.1-8B-instruct	0	一
LSTM^[25](A*)	31.0	一
MarioGPT^[7](A*)	84.0	一
LlaMario_NoAIPrompt_40K_e3(A*)	72.5	一
LlaMario_NoAIPrompt_200K_e1(A*)	84.0	一
LlaMario_AIPrompt_40K_e3(A*)	78.5	一
LlaMario_AIPrompt_40K_e5(A*)	88.5	一
Llama2-7B_AIPrompt_40K_e5(A*)	80.5	一
MarioGPT^[7](人工筛查)	88.0	4.0
LlaMario_NoAIPrompt_40K_e3 (人工筛查)	84.0	11.5
LlaMario_NoAIPrompt_200K_e1 (人工筛查)	90.0	6.0
LlaMario_AIPrompt_40K_e3 (人工筛查)	87.0	8.5
LlaMario_AIPrompt_40K_e5 (人工筛查)	94.0	5.5