- Ukusebenzisa isango lokuhlola elinezingqimba eziningi elihlanganisa ukuhlola okuqinisekile, abahluleli be-LLM, kanye nokubuyekezwa kwabantu ukuvimbela ukuzulazula kwekhwalithi.
- Ukunciphisa ukuhlukahluka okungenqunyelwe ngokusebenzisa i-N-run averaging kanye ne-gating kuma-deltas e-regression esikhundleni se-absolute thresholds.
- Ukwakha amapayipi ngendlela yesivinini esikabili lapho ukuhlolwa okusheshayo kusebenza khona kuzo zonke izibopho kanye nama-full suites asebenza ngesikhathi sokuhlanganiswa kanye nemijikelezo yasebusuku.
- Ukusebenzisa amamethrikhi e-Directed Acyclic Graph (DAG) ukuguqula izindlela eziyinkimbinkimbi, ezizimele zibe yizihlahla zezinqumo ezihlelekile neziqinisekile.
Ukuthola i-LLM ukuthi isebenze ku-demo kuyinto elula, kodwa ukuyigcina izinzile ekukhiqizeni yilapho izinto ziba zimbi khona. Amaqembu amaningi awela ogibeni "lobunjiniyela obusekelwe ku-vibe," lapho eshintsha khona i-prompt, ayisebenzise izikhathi ezimbalwa, bese ecabanga ukuthi ingcono ngoba umphumela ubukeka ulungile. Kodwa-ke, ama-LLM ayaziwa ngokuhluleka , futhi ukuthembela ekuhlolweni okumbalwa ngesandla kuyindlela yokulungisa inhlekelele uma uthumela kubasebenzisi abayizinkulungwane.
Ukuze udlule ngempela ekuqageleni, udinga ukuphatha ukuhlolwa kwakho kwe-AI njengezakhamuzi zesigaba sokuqala zengqalasizinda yakho ye-CI/CD. Lokhu kusho ukusuka ekuqhathaniseni okulula kwezintambo bese udlulela emasangweni okuhlola anezingqimba angabamba ukukhukhuleka nokubuyela emuva ngaphambi kokuba afike ekukhiqizweni. Kumayelana nokuguqula inqubo yokukholelwa ezeni ibe uhlelo lokulawula oluqinile, olungokwesayensi oluqinisekisa ukuthi imodeli yakho ayisebenzi kanye kuphela, kodwa ihlala ithembekile ngokuhamba kwesikhathi.
Ukwakheka Kwesango Lokuhlola le-CI/CD
Isango lokuhlola elifanele akuyona nje iskripthi esisodwa; liyindawo yokuhlola eguquliwe enquma ukuthi ushintsho ku-prompt, imodeli, noma isethi yedatha luphephile yini ukukhushulwa. Ngokungafani nokuhlolwa kweyunithi yendabuko okungokwesibili (kudlule noma kuhluleke), imiphumela ye-LLM iwukusatshalaliswa . Lokhu kusho ukuthi ukufaka okufanayo kungaveza imiphumela ehlukene kuyo yonke imisebenzi, okwenza umsebenzi wesango ube nzima kakhulu. Ukuze kusingathwe lokhu, uhlelo kufanele lubekwe ngezingqimba, kuqalwe ngokuhlolwa okuqinisekile (njengokuqinisekiswa kwe-schema noma i-regex), kulandelwe ijaji le-LLM eliphethwe, bese ekugcineni, kuthathwe isampula yokubuyekezwa komuntu.
Ukuze lolu hlelo lube nokwethenjelwa, yonke into eguquguqukayo kumele ifakwe. Awukwazi ukusho nje ukuthi “GPT-4”; udinga indawo yokuhlola imodeli eqondile, izinga lokushisa, kanye nembewu . Okufanayo kusebenza kumasethi akho edatha ayisisekelo kanye namarubrikhi asetshenziswa abahluleli bakho. Uma kwenzeka ukwehluleka, kufanele ukwazi ukuphinda udale indawo eqondile ukuze uqonde ukuthi kungani imodeli ibuyele emuva. Izinto zobuciko ezihunyushiwe ziyindlela kuphela yokuqinisekisa ukuthi “ukudlula” namuhla kusho into efanayo “nokudlula” ngenyanga ezayo.
Ukusebenzisa i-Four-Loop Pipeline
Ukusebenzisa i-suite yokuhlola ephelele kukho konke ukuzibophezela kuyindlela enhle yokudlulisa isabelomali sakho se-API futhi ukhungathekise abathuthukisi bakho ngezikhathi zokwakha ezisheshayo. Esikhundleni salokho, ipayipi lesivinini esikabili lisebenza kahle kakhulu. I-loop yokuqala yenzeka esigabeni se-Pull Request, igxile "kucezu lomngcele" - isethi encane, eqinile yedatha eyenzelwe ukuqala izindlela zokwehluleka ezaziwayo. Lokhu kugcina ukubambezeleka kungaphansi kwemizuzu emithathu , kuqinisekisa ukuthi onjiniyela abagcini nje ngokudlula izivivinyo.
I-loop yesibili iqala lapho ihlangana negatsha eliyinhloko. Yilapho ukudlula kweqiniso eliphelele kwenzeka khona, kuhlola yonke imingcele ye-rubric. Okwesithathu, ukuthunyelwa kwe-canary kuhambisa ingxenye encane yethrafikhi ebukhoma enguqulweni entsha, kuyibeka ngesikhathi sangempela. Okokugcina, i-loop yokukhiqiza eqhubekayo iqapha ukuzulazula okuthule , ibamba izinkinga ezibangelwa izinguquko zomhlinzeki ohamba phambili noma ukushintsha kokuziphatha komsebenzisi okungenzeka ukuthi amasethi akho edatha aqinile awazange akuthole.
Ukuxazulula Inkinga Yomsindo nge-N-Run Averaging
Esinye sezibulala ezinkulu ze-CI trust "ukuhlolwa okungacacile." Uma unjiniyela ephinda enze i-eval ehlulekile futhi idlula ngokuzumayo, uzoqala ukuphatha konke ukuhlola okubomvu njengomsindo. Lokhu kwenzeka ngoba kokubili i-LLM kanye nejaji akunqunyelwe. Isixazululo ukuyeka ukugoba emikhawulweni eqondile . Ukubeka umugqa oqinile ku-0.85 akunamsebenzi uma indawo yakho yomsindo yemvelo ingu-0.04; ukugijima okukodwa kungafika nhlangothi zombili zalowo mugqa ngenhlanhla nje.
Kunalokho, kufanele usebenzise icala ngalinye lokuhlola izikhathi ezingu-N (ngokuvamile u-3 kuya ku-5) bese ubala isilinganiso samaphuzu angu- . Okubaluleke kakhulu, kufanele uvale i -delta ngokumelene nesisekelo . Yehluleka ukwakha kuphela uma ukuhlehla kudlula ukubekezelelana okuthile—ngokuvamile ukuphambuka okujwayelekile okubili kwebhendi yomsindo. Ngokuqhathanisa imiphumela ye-PR ngokumelene nesisekelo esisanda kulinganiswa emsebenzini ofanayo, ukhansela ukuhlukahluka okwabiwe, uqinisekise ukuthi “ukwehluleka” empeleni kubonisa ukwehla kwangempela kwekhwalithi.
I-Determinism Ethuthukisiwe nge-DAG Metrics
Nakuba i-LLM-njengomahluleli inamandla, ingaba yinto ethinta kakhulu imisebenzi eyinkimbinkimbi. Isibonelo, ukuhlola ukuthi isifinyezo sinesingeniso, umzimba, kanye nesiphetho ngokulandelana okufanele kunzima ukuthi isiphakamiso esisodwa sithole amaphuzu ngokuqhubekayo. Yilapho kungena khona amamethrikhi e-Directed Acyclic Graph (DAG) . Esikhundleni sesiphakamiso esisodwa esikhulu, uhlukanisa ukuhlolwa kube umuthi wesinqumo onezinhlobo ezithile zama-node: ama-node emisebenzi yokukhipha, ama-node okwahlulela amabili wokuhlola okungu-yebo/cha, kanye nama-node okwahlulela amaphuzu okugcina.
Le ndlela ehlelekile yenza ukuhlolwa kube okuqinisekile ngokuphelele futhi ivumela ngisho namamodeli amancane, ashibhile ukuthi enze ukwahlulela kwekhwalithi ephezulu ngoba aphatha kuphela amayunithi e-athomu omsebenzi. Futhi kwenza ukulungisa amaphutha kube lula; ungalandelela indlela eqondile ukuhlolwa okuthathwe ngayo kugrafu ukuze ubone ukuthi kwehluleke kuphi. Ngokuhlanganisa lawa ma-DAG nokufana kwencazelo esikhundleni sokufanisa izintambo ngqo, ugwema ukumaka izimpendulo ezifanele ngoba nje amagama ahlukile kancane kulokho okubhekiselwe kukho.
Ukusebenza kwesethi yedatha
Ukuhlola kwakho kuhle njengedatha yakho kuphela. Iphutha elivamile ukufaka ikhodi "yezimpendulo zegolide" kumafayela okuhlola, okwenza angabonakali ekubuyekezweni kontanga. Indlela engcono ukugcina amasethi edatha njengamafayela e -JSON Lines aguquliwe endaweni yakho yokugcina idatha. Lokhu kuvumela ukuthi konke okungeziwe noma ushintsho kulandelwe ngesicelo sokudonsa. Ukuze uvimbele isethi yedatha ukuthi ingaphelelwa yisikhathi, kufanele unikeze njalo ukwehluleka kokukhiqiza kanye namasampula abuyekezwe ngabantu emuva eqinisweni eliyisisekelo.
Uma idatha ingekho, ungalindi ukuqoqa izinkulungwane zezibonelo. Gxila ebubanzini bobukhulu kunevolumu. Amacala ambalwa akhethwe kahle ahlola isisekelo, ukulandela imiyalelo, kanye nokuphepha angabamba izigaba eziphelele zokwehluleka. Ngokugxila kumacala omngcele — lawo ahlala onqenqemeni lokudlula nokwehluleka — wandisa ubukhulu besiginali se-suite yakho ngaphandle kokudinga isethi yedatha enkulu.
Ukuguqula ukuhlolwa kwe-LLM kusuka emsebenzini wesandla kuya enkambisweni ye-CI enzima kudinga ushintsho endleleni yokucabanga kusukela ekuqinisekisweni kwe-binary kuya ekusatshalalisweni kwezibalo. Ngokusebenzisa amapayipi ahlukene, isilinganiso sokugijimisa okuningi ukuqeda umsindo, kanye nokusebenzisa izilinganiso ze-DAG ezihlelekile, amaqembu angathumela izinguquko ezisheshayo kanye nokuthuthukiswa kwemodeli ngokuzethemba kwangempela. Umgomo awukona ukwenza i-LLM ibe yinto eqinisekile, kodwa ukwenza isango lokuhlola libikezelwe , ukuqinisekisa ukuthi ukuhlolwa okuluhlaza konke okusemgwaqweni kumele izinga lekhwalithi eliqinisekisiwe.
