﻿# 2026.5.8-鏃ユ姤-寮犲鎸痋n
## 宸ヤ綔鍐呭姒傝堪

1銆佸涔燣oRA杩涜妯″瀷寰皟

---

<callout emoji="馃">
浠婂ぉ鐪嬪埌Google涓?*Gemma 4鎺ㄥ嚭澶?Token 棰勬祴 (Multi-Token Prediction, MTP) 鑽夌鍣?*锛岃兘灏嗘帹鐞咷emma 4閫熷害鎻愬崌涓ゅ埌涓夊€嶃€傝胺姝屽凡缁忔湁浜咷emini杩欐牱鐨勫ぇ妯″瀷锛屼负浠€涔堣繕瑕佷负Gemma 4涓嬭繖涔堝ぇ鍔熷か銆傛垜鐨勬兂娉曟槸灏忔ā鍨嬬殑鏍稿績浠峰€兼鍦ㄤ簬鈥?*鍙繁搴﹀畾鍒?*鈥濄€侴emma杩欑被寮€婧愬熀搴э紝鐪熸鐨勪綔鐢ㄦ槸琚紑鍙戣€呮嬁鍘讳簩娆¤捀棣忔垨寰皟锛屾墦纾ㄦ垚閫傞厤鑷韩鍨傜洿涓氬姟鐨勪笓灞炴ā鍨嬶紝鑰孧TP杩欑被鍔犻€熸妧鏈垯璁╀笟鍔¤惤鍦版椂鍝嶅簲鏇村揩銆佹垚鏈洿浣庘€斺€斾粠Gemini娴撶缉鍒癎emma锛屽啀浠嶨emma钂搁鍑虹粏鍒嗗満鏅殑涓撳銆俓n</callout>

## **涓€銆丩oRA鎶€鏈?*

LoRA 鏄竴绉嶈交閲忕骇鐨勫ぇ妯″瀷寰皟鎶€鏈紝鏍稿績鎬濇兂鏄湪鍘熷棰勮缁冩潈閲嶆梺杈归檮鍔犱綆绉╃煩闃碉紝浠呰缁冭繖浜涘皬鍙傛暟锛岃€屼笉鍔ㄥ師濮嬫ā鍨嬨€俓n
**锛堜竴锛夊伐浣滃師鐞?*

- 妯″瀷鍏ㄩ噺寰皟鏃讹紝闇€瑕佹洿鏂板叏閮ㄦ潈閲嶇煩闃?$W$锛屾樉瀛樺拰璁＄畻寮€閿€鏋佸ぇ銆俓n- LoRA 鐨勬妸鎴忥細鏉冮噸鏇存柊閲?$螖W$鐢ㄤ袱涓皬鐭╅樀 $B$ 鍜?$A$ 鐨勪箻绉潵杩戜技锛屽嵆 $螖W=BA$锛屽叾涓?A,B*A*,*B* 閮芥槸浣庣З鐭╅樀锛屽弬鏁伴噺杩滃皬浜庡師濮?$W$銆俓n- 鍓嶅悜璁＄畻鍙樹负锛?h=Wx+BAx$銆傝缁冩椂鍐荤粨 $W$锛屽彧鏇存柊 $A$ 鍜?$B$銆俓n
**锛堜簩锛夎兘骞蹭粈涔?*

- **鏋佷綆鎴愭湰寰皟澶фā鍨?*锛?鍦ㄥ崟涓秷璐圭骇鏄惧崱涓婂氨鑳藉井璋?7B銆?3B 鐢氳嚦鏇村ぇ鐨勬ā鍨嬨€俓n- **澶氫换鍔″揩閫熷垏鎹?*锛?涓嶅悓浠诲姟鍙渶淇濆瓨鍜屽姞杞戒笉鍚岀殑 LoRA 鏉冮噸鏂囦欢锛堥€氬父鍑?MB 鍒板嚑鍗?MB锛夛紝鏃犻渶涓烘瘡涓换鍔″瓨涓€浠藉畬鏁存ā鍨嬨€俓n- **棰嗗煙閫傞厤涓庝釜鎬у寲**锛?鐢ㄥ皯閲忛鍩熸暟鎹嵆鍙閫氱敤澶фā鍨嬪彉鎴愨€滆涓氫笓瀹垛€濓紝濡傚尰鐤楅棶绛斻€佹硶寰嬫枃涔︺€佹父鎴?NPC 浜烘牸鍖栫瓑銆俓n- **淇濇姢鍘熸ā鍨嬭兘鍔?*锛?涓嶇牬鍧忓熀搴фā鍨嬬殑閫氱敤鐭ヨ瘑锛屽彲闅忔椂鎻掓嫈锛屽嚑涔庢棤鐏鹃毦鎬ч仐蹇橀闄┿€俓n- **缁撳悎鐭ヨ瘑钂搁**锛?鍙笌鈥滃ぇ妯″瀷鏁欏皬妯″瀷鈥濇祦绋嬪ぉ鐒堕厤鍚堬紝璁╁皬妯″瀷鍦ㄧ壒瀹氫换鍔′笂浠ユ瀬灏忎唬浠风户鎵垮ぇ妯″瀷鑳藉姏銆俓n
## **浜屻€佺粨鍚堢煡璇嗚捀棣忕殑濂藉**

1. **涓€涓€氱敤澶фā鍨嬶紝澶嶅埗鍑哄涓笓鐢ㄥ皬妯″瀷**

鐢ㄥぇ妯″瀷閽堝涓嶅悓浠诲姟鍒嗗埆鐢熸垚璁粌鏁版嵁锛屾暀鍑哄鏈嶅皬妯″瀷銆佹憳瑕佸皬妯″瀷銆佺紪绋嬪皬妯″瀷绛夈€傛瘡涓皬妯″瀷鍙仛涓€浠朵簨锛屾晥鏋滄洿绋炽€佹洿濂借皟浼樸€俓n
1. **闄嶄綆瀵归珮绔‖浠剁殑渚濊禆**

涓嶆槸鎵€鏈夊満鏅兘閫傚悎閰嶅鏄傝吹鐨勬湇鍔″櫒闆嗙兢锛屽皬妯″瀷璁╃畻鍔涙湁闄愮殑鐜涔熻兘鐢ㄤ笂濂界敤鐨?AI銆俓n
1. **鎴愭湰澶у箙闄嶄綆**

澶фā鍨嬭窇涓€娆￠渶瑕佺殑鏄惧崱鍜屾椂闂磋繙澶氫簬灏忔ā鍨嬨€傚悓鏍风殑鏈嶅姟锛屾崲灏忔ā鍨嬭兘璁╂湇鍔″櫒寮€閿€闄嶅埌鍑犲崄鍒嗕箣涓€銆傚鏋滄嬁API鏉ヨ皟鐢紝澶фā鍨嬭皟鐢ㄦ垚鏈瘮杈冮珮锛屼笉閫傚悎浣庢敹鍏ョ殑鍦烘櫙锛岃€屽皬妯″瀷鍙互鐩存帴璺戝湪鏈湴鏈嶅姟鍣ㄥ氨琛岋紝瀵规湇鍔″櫒鐨勮姹備篃娌℃湁閭ｄ箞楂樸€俓n
1. **鍝嶅簲閫熷害鎴愬€嶆彁鍗?*

澶фā鍨嬬敓鎴愪竴娈佃瘽鍙兘瑕佸ソ鍑犵锛屽皬妯″瀷寰€寰€涓嶅埌涓€绉掋€傜敤鎴风瓑寰呮椂闂寸煭锛屽悓鏃惰兘鏈嶅姟鏇村鐨勭敤鎴枫€俓n
## 涓夈€佸涔犲疄璺礬n
涓轰簡寰皟鐨勯€熷害锛岄€夌敤鐨勬ā鍨嬪熀搴ф槸`gemma-3-1b`锛岄€氳繃LM Studio杩涜璋冪敤

**锛堜竴锛夋簮鐮佹暣鐞嗕笌绉嶅瓙鏋勫缓**

- 閫氳繃寮€婧愰」鐩娊鍙栨簮鐮佺墖娈?鏂囦欢锛岀敓鎴愯缁冪瀛愭彁绀鸿瘝锛岀洰鏍囨槸璁╁悗缁暀瀛﹀唴瀹圭揣璐翠笟鍔′唬鐮併€俓n
**锛堜簩锛夐€氳繃 MiniMax 杩涜鏁欏鏁版嵁鐢熸垚**

- 璋冪敤 MiniMax API锛屽姣忎釜婧愮爜鏍锋湰鐢熸垚楂樿川閲忚瑙ｄ笌鏀硅繘寤鸿銆俓n- 灏嗐€屾簮鐮? MiniMax 璁茶В銆嶆嫾鍚堟垚鐩戠潱鏁版嵁锛屽舰鎴?`train.jsonl`銆俓n- 瀵逛綆璐ㄩ噺鏍锋湰鍋氭竻娲楋紝鎻愬崌璁粌鏁版嵁瀵嗗害鍜屽噯纭€с€俓n
**锛堜笁锛塙nsloth LoRA 寰皟璁粌**

- 浣跨敤 `unsloth/gemma-3-1b-it-unsloth-bnb-4bit` 杩涜寰皟銆俓n- 鏈疆鍦ㄥ凡鏈夋潈閲嶅熀纭€涓婄画璁紙`gemma-lora-v1 -> gemma-lora-v2`锛夛紝鎻愰珮瀵归」鐩唬鐮侀鏍间笌瑙ｉ噴鑳藉姏鐨勯€傞厤銆俓n
**锛堝洓锛夎缁冪粨鏋滀笌妯″瀷浜х墿**

- 鎴愬姛浜у嚭鏂扮殑 LoRA 閫傞厤鍣ㄦ潈閲嶏紙`gemma-lora-v2`锛夈€俓n- 璁粌 loss 鏀舵暃姝ｅ父锛岃繘鍏ラ儴缃茶浆鎹㈤樁娈点€俓n
**锛堜簲锛夐儴缃查摼璺噯澶囷紙鏈湴LM Studio璋冪敤锛?*

- 鎸夋祦绋嬫墽琛岋細LoRA 鍚堝苟鍏ㄩ噺妯″瀷 -> 杞?GGUF(F16) -> 閲忓寲(Q4_K_M) -> 瀵煎叆 LM Studio銆俓n
![](https://internal-api-drive-stream.feishu.cn/space/api/box/stream/download/authcode/?code=OGJmYjljMzcyOWIwNmRkNDcyOTBmYjg4NTJmYjE0MGZfNjhjMWRiNTllNTkzMmQzNjQxMTFhYmYzZTI2ZTNjZmFfSUQ6NzYzNzUyNDA1MjQzMzI5MjIxNl8xNzc5MTc2MTI3OjE3NzkxNzk3MjdfVjM)

![](https://internal-api-drive-stream.feishu.cn/space/api/box/stream/download/authcode/?code=M2ZmNTc4N2FlNjY3YTZhNWQwZDM4YjcyMjM1Y2Q1NjlfZTY4NTYzOTJmMDRmZDFkNTdkNWQyZGQ1MDczN2ZmMzFfSUQ6NzYzNzUzMTg0NzIwNzA0NjM1OV8xNzc5MTc2MTI3OjE3NzkxNzk3MjdfVjM)

<callout emoji="馃槃">
璇曚笅鏉ユ劅瑙夛紝1B妯″瀷鐨勫簳瀛愮‘瀹炲亸钖勶紝鐢熸垚鍐呭杩樻槸瀹规槗鈥滆儭瑷€涔辫鈥濄€備笉杩囩粡杩囦竴娈垫椂闂村涔犲悗锛屽湪杈撳嚭鏍煎紡鍜岄鏍间笂鏄庢樉鍚戝畠鐨勨€滃笀鍌呪€滿inimax闈犳嫝浜嗐€俓n</callout>
