Ama-ejenti e-AI endawo ku-ESP32: izinhlaka, abasizi bezwi kanye namaphrojekthi angempela

Ikhaya » Python » Ama-ejenti e-AI endawo ku-ESP32: izinhlaka, abasizi bezwi kanye namaphrojekthi angempela

Ukusebenzisa ama-ejenti e-AI endawo ku-ESP32 akuseyona inganekwane yesayensi noma into yokuzilibazisa eyingqayizivele yabaduni behadiwe abanamandla. Phakathi kwezinhlaka ezifana ne-ESP Claw, i-PycoClaw, ama-hybrid voice assistant stacks asebenzisa i-LangChain noma i-MCP, kanye namaphrojekthi e-DIY angempela, i-ESP32 ecosystem isiphenduke buthule yaba yindawo yokudlala engathi sína yobuhlakani obuphezulu. Manje usungakwazi ukwakha amadivayisi alalela, anqume futhi enze izinto ezweni elingokoqobo ngenkathi ebiza amadola ambalwa futhi esebenza ngisho nangokuxhumeka okungabonakali.

Lo mhlahlandlela ugxila kakhulu kulokho okushiwo ngempela ukubamba ama-ejenti e-AI ku-ESP32, ukuthi amafreyimu afana ne-ESP‑Claw kanye ne-PycoClaw abhekana kanjani nenkinga, lapho ama-backend e-cloud esakhanya khona, nokuthi yiziphi izimo zokusebenzisa ezinengqondo ngempela kuhadiwe enjalo evinjelwe. Sizophinde sihambe ngezakhiwo ezingokoqobo zabasizi bezwi, ukuzenzakalela kwekhaya, ukuqapha kwezimboni ngisho namaphrojekthi adlalayo njenge-cyberpet kanye nabalingiswa abaphathekayo, konke kuqhutshwa yizilawuli ezincane kodwa ezinekhono elimangalisayo.

Kungani i-AI isuka efwini iye emaphethelweni

Eminyakeni embalwa edlule, i-AI isiqalile ukushintsha indlela yokucabanga emsulwa ethi “konke okusefwini” iye kumodeli ehlanganisiwe lapho ubuhlakani buhlala khona eduze kakhulu nomthombo wedatha. Ku-IoT lo mkhuba usobala: abathuthukisi bafuna ukunciphisa ukubambezeleka, bagweme ukuthumela idatha ebucayi kumaseva ezinkampani zangaphandle futhi bagcine ukusetshenziswa kwamandla kulawulwa. Uhambo oluqhubekayo oluya efwini luyabiza, luyahamba kancane futhi, kweminye imikhakha, alwamukeleki nhlobo ngokombono wobumfihlo noma wokuthobela imithetho.

Kulesi simo, amadivayisi ekilasi le-ESP32 aba “ama-node ahlakaniphile onqenqemeni” esikhundleni sokuba ama-data forwarder ayiziwula. Iphethini evamile namuhla ukuvumela i-microcontroller ukuthi isebenzise amamodeli alula kanye nama-ejenti asekelwe emithethweni endaweni, iphatha ukuhlanganiswa kwezinzwa, ukusebenza kanye nezinqumo zesikhathi sangempela, ngenkathi ilayisha izinto ezisindayo (ukuqashelwa kwenkulumo okugcwele, ukucabanga okukhulu, izimpendulo zokukhiqiza) kuma-LLM efu kuphela uma kudingeka.

Izinhlaka ezifana ne-ESP‑Claw kanye ne-PycoClaw zingena kahle kulesi sithombe esihlanganisiwe. Abazami ukufaka imodeli yolimi enkulu egcwele kwisabelomali se-RAM esingu-520 KB; kunalokho, bahlela amamodeli amancane, agxile kanye ne-deterministic logic engasebenza kudivayisi, futhi ngokuzikhethela bakhuluma nezinsizakalo zamafu lapho umsebenzi udinga amandla ehhashi engeziwe. Inzuzo ukubambezeleka okuphansi, ukusebenza okuqinile kumanethiwekhi angasebenzi kahle kanye nokulawula okuqinile kakhulu ukuthi iyiphi idatha ephuma kudivayisi.

Ezimweni zokusetshenziswa njengekhaya elihlakaniphile, ukuzenzekela okulula kwezimboni noma ezolimo, leli cebo lokuqala liyakhanga kakhulu. Izibani kumele zisabele ngokushesha ekunyakazeni, imigqa yokukhiqiza ayikwazi ukuma ngoba i-inthanethi ayisebenzi, futhi amapulazi akude awakwazi ukuthembela ekuxhumekeni kwamaselula amahora angama-24 ngosuku, izinsuku eziyi-7 ngesonto. Ama-ejenti e-AI endawo ku-ESP32 avumela lezi zinhlelo ukuthi ziqhubeke zisebenza - futhi zivame ukusebenza kangcono - ngisho noma ifu lingafinyeleleki.

I-ESP32 njengeplatifomu ye-AI: amandla kanye nemikhawulo enzima

Umndeni we-ESP32 uthole idumela lawo emhlabeni womenzi kanye nochwepheshe ngokuhlanganisa i-Wi‑Fi, i-Bluetooth kanye ne-computing enhle ngentengo ephansi kakhulu. I-ESP32 ejwayelekile inikeza i-CPU ye-Xtensa ekabili efinyelela cishe ku-240 MHz, cishe i-520 KB ye-SRAM, ama-megabyte amaningana e-flash, kanye ne-PSRAM eyengeziwe enweba imemori esebenzisekayo yemithwalo yemisebenzi enzima kakhulu.

Ngokombono we-AI, le hardware ngokusobala incane uma iqhathaniswa nama-GPU noma ngisho nama-smartphone anamuhla, kodwa isanele kumamodeli alungiselelwe ngokucophelela kanye ne-ejenti logic. Ungaqhuba kahle amanethiwekhi amancane e-neural ngemisebenzi efana nokubona amagama angukhiye, ukuhlukaniswa komsindo okuyisisekelo, ukutholwa okulula kwe-anomaly kudatha yenzwa noma izinqubomgomo zesinqumo eziqondile ezihlanganisa okokufaka okuningi.

Ukusetshenziswa kwamandla kungenye iphuzu elinamandla le-ESP32. Kumodi esebenzayo ivame ukudonsa i-ballpark engu-80-260 mA ku-3.3 V (cishe u-0.3-0.85 W), futhi i-chip inikeza isethi ecebile yezindlela zokulala. Uma i-AI isebenza endaweni, wonga amandla abengasetshenziswa ukudlulisa idatha eluhlaza ngokuqhubekayo efwini, futhi ungavusa idivayisi kuphela lapho injini yomodeli noma yomthetho inquma ukuthi kukhona okuthakazelisayo okwenzekayo.

Izindleko zingase zibe yisici esiphazamisa kakhulu: amabhodi amaningi asekelwe ku-ESP32 athengiswa ngama-euro angaphansi kwe-10, amanye acishe abe amadola ama-5 ngobuningi. Lokho kukuvumela ukuthi usakaze amashumi noma amakhulu ama-node ahlakaniphile endlini, phansi efektri, ensimini noma endaweni yokuthengisa ngaphandle kokunciphisa isabelomali. Uma kuqhathaniswa nama-edge gateways noma ama-PC ezimboni, i-bill yezinto zokwakha iphansi kakhulu.

Uhlangothi oluphambene luwukuthi inkumbulo kanye nophahla lokubala kungokoqobo kakhulu futhi kuzobumba zonke izinqumo zakho zokuklama. Njengoba kungaphansi kwe-1 MB etholakalayo kumamodeli asethelweni avamile, kufanele wamukele amasu afana nokulinganisa okungu-8-bit, ukuthena okunamandla, ukunciphisa amapharamitha kanye nokusebenza okuqhubekayo. Noma yini efana ne-LLM yesimanje esetshenziswayo ayifanele; ongabamba esikhundleni salokho amamodeli amancane, ahlelwe kahle kanye nezihibe ze-agent ezibiza izinsizakalo zangaphandle zokucabanga okunzima lapho kudingeka.

I-ESP Claw: ama-ejenti alula kudivayisi e-ESP32

I-ESP‑Claw, eyathuthukiswa yi-Espressif Systems, iwuhlaka oluklanyelwe ngqo ukusebenzisa ama-AI agent endawo ngqo kuma-microcontroller e-ESP32. Esikhundleni sokuphatha idivayisi njengeklayenti elincane elithumela konke efwini, i-ESP Claw iyiguqula ibe yinjini encane yokwenza izinqumo engafunda izinzwa, isebenzise ukuqagela futhi ishayele ama-actuator ngokwayo.

Ngaphansi kwe-hood, i-ESP‑Claw isebenzisa ukwakheka kwe-modular okunamabhlogo amathathu okwakha ayinhloko: injini yokuqagela elula, ungqimba lokuphatha i-ejenti kanye nama-hook okuhlanganisa ama-sensor nama-actuator. Abathuthukisi bachaza ama-ejenti njengezinhlangano ezithola okokufaka, zikucubungule ngemodeli ehlanganisiwe kanye nesethi yemithetho, bese zikhipha okukhiphayo okubangela izenzo ezifana nokuguqula ama-relay, ukuthumela izexwayiso noma ukulungisa ama-control setpoints.

Ngenxa yokuthi i-RAM ilinganiselwe kakhulu, i-ESP‑Claw incike kakhulu kumamodeli amancane kanye nokulungiswa kwe-ML okuhlanganisiwe okuvamile. Amasu ajwayelekile afaka phakathi i-quantization engu-8-bit, ukunqunywa kwamapharamitha kanye nokuqalisa ukuphetha ngezinyathelo ezincane ukuze ama-buffer aphakathi afaneleke kwimemori. Umphumela osebenzayo ukuthi ungabamba amamodeli angaphansi kwe-1 MB asafinyelela ukunemba okungu-80-90% emisebenzini yokuhlela eyisisekelo, okwanele engxenyeni enkulu yezimo ze-IoT.

Ukubambezeleka yilapho le ndlela yendawo ikhanya khona ngempela. Ucingo olujwayelekile lwamafu lungathatha ama-ms ayi-100-500 kuye ngenethiwekhi, okungaba yingozi kuma-loop okulawula okuqinile noma ama-interface abasebenzisi aphendulayo. Nge-ESP‑Claw, ukuqagela okulula kuvame ukuphelela ngaphansi kwama-ms ayi-10, okuvumela ukuzenzekela kwesikhathi sangempela emigqeni yezimboni, izinhlelo zokuphatha izakhiwo noma ukufakwa okusebenzisanayo.

I-ESP‑Claw iphinde isekele ukuxhumana nge-Wi‑Fi ne-Bluetooth, ngakho amadivayisi angakwazi ukubika izifinyezo, ukuthumela amalogi noma ukuthola izibuyekezo uma inethiwekhi itholakala. Kodwa-ke, isiphakamiso esiyinhloko senani ukuthi i-ejenti iyaqhubeka nokusebenza ngokuzimela ngisho noma lokho kuxhumana kunyamalala, igcina ubumfihlo kanye nokuqina.

I-PycoClaw: Ama-ejenti esitayela se-OpenClaw ku-ESP32 nge-MicroPython

Nakuba i-ESP‑Claw igxile ku-C/C++ kanye namamodeli amancane, i-PycoClaw ithatha uhlangothi oluhlukile ngokuletha ukwakheka kwe-ejenti ye-OpenClaw ku-ESP32 nge-MicroPython. Umgomo uzimisele kakhulu: vumela i-microcontroller yamadola amahlanu isebenzise ama-ejenti okukhiqiza anememori, amathuluzi kanye nokuhlelwa kweziteshi eziningi okubukeka njenge-backend stack yesimanje - imane yehliswe kakhulu.

I-OpenClaw ngokwayo iwuhlaka lomthombo ovulekile olwenzelwe ukwakha ama-ejenti e-AI athembekile, alawulekayo kusetshenziswa iphethini ye-hub ne-spoke. Esikhundleni sokumane ugoqe i-LLM, inikeza ipayipi elihlelekile elinezigaba eziyisithupha: ukungenisa, ukuhambisa, ukuhlanganisa umongo, ukushayela imodeli, ukusebenzisa ithuluzi kanye nokulethwa kwempendulo. I-ejenti ngayinye inesikhala sokusebenza esihlukanisiwe esinamafayela ombhalo ocacile njenge-AGENS.md, i-SOUL.md kanye ne-USER.md echaza ubuntu bayo, imithetho kanye nomongo womsebenzisi.

I-PycoClaw ivumelanisa le filosofi ne-MicroPython ku-ESP32, ifaka imisebenzi eminingi ezinsizeni ezilinganiselwe. Iza ne-IDE efinyeleleka kalula yesiphequluli ephatha ukukhanya kwe-firmware kanye nokusethwa kwendawo, ukuze abasunguli abangebona ochwepheshe bakwazi ukufaka ibhodi, bachofoze inkinobho bese bethumela i-ejenti ngaphandle kokulwa nama-toolchain noma ama-Makefiles.

Esinye sezici ezinhle kakhulu ze-PycoClaw ukufinyelela ngqo kuma-interface e-hardware ngaphakathi kwe-logic ye-agent. Ama-ejenti asebenza ku-MicroPython angakhuluma ngokwendabuko ne-GPIO, i-I2C, i-SPI kanye ne-PWM, okusho ukuthi into efanayo exoxa, ebiza amathuluzi noma ebuza ama-API ingafunda nezinzwa, ishayele ama-motor, ibuyekeze izibonisi noma i-flip relay ngaphandle kwesendlalelo sebhuloho esibuthakathaka phakathi.

Ngasohlangothini lokuxhumana, i-PycoClaw ikhombisa imodeli yengxoxo yeziteshi eziningi ye-OpenClaw ngaphakathi kwe-microcontroller. I-ESP32 eyodwa ingaphatha imiyalezo nge-Bluetooth, i-Wi‑Fi, i-serial noma i-MQTT, idlulise konke ngesikhathi esifanayo sokusebenza kwe-ejenti. Lokho kwenza kube lula kakhulu ukusekela uhlelo lokusebenza lweselula, ideshibhodi yewebhu kanye nomthengisi wezimboni ngesikhathi esisodwa, ngaphandle kwekhodi yokuhlanganisa ngokwezifiso ngesiteshi ngasinye.

Inkumbulo, ukuphikelela kanye ne-ScriptoHub ohlelweni lwe-PycoClaw

Lapho imitapo yolwazi ye-ML ehlanganisiwe yakudala iphelela ekucabangeni, i-PycoClaw igcizelela kakhulu ukuphathwa kwesimo kanye nenkumbulo eqhubekayo. Isimo se-ejenti – amaseshini, izintandokazi, amanothi, imininingwane yomuntu siqu – kugcinwa ku-flash ye-ESP32 kusetshenziswa izinhlelo zamafayela ezifana ne-SPIFFS noma i-LittleFS, ngakho idivayisi igcina umongo kuyo yonke i-reboot, imijikelezo yamandla kanye nokuphazamiseka kwenethiwekhi.

Lokhu kuqhubeka akuyona nje isici esihle se-UX; ekusetshenzisweni kwezimboni nasemasimini kuba yimfuneko enzima. Abaqhubi balindele ukuthi ama-ejenti akhumbule ama-alamu adlule, izinguquko zokucushwa kanye nokweqiwa kwendawo, futhi abahloli bokuthobela imithetho bavame ukufuna izinkomba ezicacile zezinqumo. Ukugcina lokhu kudivayisi esikhundleni sokuphinda udonse konke kusuka ku-backend yamafu kusiza ukugcina uhlelo luqinile ngisho nalapho ukuxhumana kungathembekile.

Ukuze kusheshiswe intuthuko, i-PycoClaw ixhuma ku-ScriptoHub, imakethe yomphakathi yezikripthi ze-ejenti ezakhiwe ngaphambilini. Lapho ungathola amamojula okwenza izinto zisebenze ekhaya, amarobhothi amancane, abasizi basensimini, amadeshibhodi e-telemetry nokuningi. Amaqembu angangenisa la makhono, awalungise ukuze avumelane nomkhiqizo wawo bese enikela ngokuthuthukisa, kancane kancane akhe uhlelo oluhlanganyelwe oluzungeze uhlaka.

Uma kuqhathaniswa nezixazululo ezisezingeni eliphansi njengeTensorFlow Lite Micro noma i-Edge Impulse, iPycoClaw inendawo ehlukile. Lawo mathuluzi asebenza kahle kakhulu ekucubunguleni ukusakazwa kwezinzwa - cabanga ngokuhlukaniswa kokudlidliza noma ukuqashelwa kwezimpawu - kodwa awanikezi inkumbulo, amathuluzi, ingxoxo yeziteshi eziningi noma umzila wezinga eliphezulu. Ngakolunye uhlangothi, izixazululo ezisindayo njenge-AWS IoT Greengrass zinikeza amakhono acebile onqenqemeni ngezindleko zamanani aphezulu ngedivayisi kanye nokuthembela okuqinile kwamafu.

Kwabaqalayo abasafufusa abakha imikhiqizo ekhaya elihlakaniphile, kumarobhothi noma ezenzakalelayo ezishibhile, i-PycoClaw stack ikhanga kakhulu. Uthola ukubambezeleka okuqinile, ukulawulwa kwehadiwe yekilasi lokuqala kanye nokuziphatha okuvezwa njengamafayela ombhalo ahlelekayo kunokuba kube yi-firmware ekhanyiswe kabusha njalo, okusheshisa kakhulu ukuhlola kanye nokuphindaphinda.

Abasizi bezwi ku-ESP32: izitaki ezihlanganisiwe ezine-LangChain, i-MCP kanye nama-LLM amafu

Ngaphandle kohlaka olujwayelekile "lwe-agent", enye yezinhlelo zokusebenza ezisetshenziswa kakhulu ze-ESP32 ukuba yisivikelo sangaphambili sabasizi bezwi. Kulezi zakhiwo, i-microcontroller iphatha i-audio I/O, i-UI eyisisekelo kanye nokulawula ihadiwe, kuyilapho imisebenzi enzima yokuqonda - ukubhala phansi, ukucabanga, ukwenziwa kwenkulumo esezingeni eliphezulu - isebenza efwini.

Ukwakhiwa okuvamile kusebenzisa i-ESP32 (ngokuvamile i-ESP32‑S3 ukuze ithole ukwesekwa okungcono komsindo) ukuze ithwebule umsindo ngemakrofoni ye-I2S, iphathe izinkinobho zokucindezela noma izinzwa zokuthinta, futhi idlale umsindo nge-amplifier ye-I2S kanye nesipikha. Umsindo ongahluziwe noma ocutshungulwe kancane usakazwa nge-WebSockets kuseva ye-backend (ngokuvamile i-Node.js/TypeScript), ehlanganisa izinsizakalo: i-Whisper noma imodeli efanayo ye-ASR, i-LLM nge-LangChain yokuqonda nokwenza impendulo, kanye nenjini ye-TTS yokukhipha umsindo.

I-backend bese ithumela umsindo ohlanganisiwe emuva ku-ESP32 ngezingcezu ezincane, okuyinto idivayisi edlala cishe ngesikhathi sangempela. Ngokombono womsebenzisi, kuzwakala sengathi “i-walkie‑talkie enobuchopho” ephendula ngokushesha nangokwemvelo, kuyilapho i-logic esindayo ihlala endaweni yeseva ekwazi ukukhuliswa futhi ethuthukiswa kalula.

Enye yemininingwane yobuchwepheshe eyinkimbinkimbi kulezi zinhlelo ukuphathwa kwe-buffer kuzo zombili izinhlangothi zokuxhumeka. Udinga ukulungisa osayizi be-buffer, amazinga okusampula kanye namasu okuhlukanisa ngokucophelela ukuze ugweme amaphutha kanye nezikhala ezinde ezimpendulweni. Ngezilungiselelo ezifanele, la maphrojekthi angafinyelela ezikhathini zokuguquka ezizwakala zibushelelezi njengengxoxo esikhundleni se-robotic kanye ne-laggy.

Ngasohlangothini lwephrothokholi, i-MCP (Model Context Protocol) kanye nezindlela ezifanayo seziqalile ukudlala indima enkulu. I-MCP ichaza indlela ejwayelekile yokuthi ama-ejenti akhangise futhi asebenzise “amathuluzi” – imisebenzi efana nokufunda inzwa, ukujika i-relay, ukubuza i-API yebhizinisi noma izibani zokulawula – ngendlela echazayo. Lokhu kuhlukanisa ukukhetha kwemodeli ye-AI kusuka ku-logic yokuhlanganiswa kwehadiwe eyisisekelo futhi kwenza kube lula kakhulu ukushintsha abahlinzeki bamamodeli ngaphandle kokubhala kabusha ikhodi yokulawula idivayisi.

Amaphrojekthi omhlaba wangempela: izilwane ezifuywayo ze-cyber, amakhophi e-Wheatley kanye nabasizi be-DIY

Konke lokhu kungase kuzwakale kungaqondakali uze ubheke amadivayisi aqinile abantu asebevele bewasebenzisa ku-ESP32. Isibonelo esisodwa esivelele "ikati" ledeskithophu yesitayela se-cyberpunk elisebenza nge-ESP32 S3 kanye nesibonisi samaphikseli angu-410 × 502. Lesi silwane esincane sisebenza njengomngane obonakalayo ovumela izwi, ngokuvumelanisa izindebe ngesikhathi sangempela, ukubonakaliswa kanye nobuntu.

Kuleyo sakhiwo, i-ejenti (evame ukusetshenziswa kusetshenziswa ukuhlelwa kwesitayela se-MCP) ixhumanisa amamojula amaningana e-AI. Ukukhishwa kwefoneme kumsindo okhiqizwe kuqhuba ipayipi lokugqwayiza komlomo elilungiselelwe ukukhiqiza ukunyakaza kwezindebe okubonakala kungokwemvelo, kuyilapho i-logic ehlukile iphatha izimpendulo, ukuziphatha okungasebenzi kanye nokusabela ekusebenzisaneni komsebenzisi. Umphumela wokugcina umlingiswa ozizwa ephila ngokwanele ukuba umdali amshiye esebenza "njengomngane" ngesikhathi semidlalo yebhodi eyedwa.

Elinye icala elijabulisayo inguqulo ephathekayo ye-Wheatley evela ku-Portal 2, esetshenziswe ku-SenseCAP Watcher (esekelwe ku-ESP32 ene-8 MB PSRAM). Lapha, i-firmware eyakhiwe nge-ESP‑IDF isebenzisa i-WebRTC ukusakaza umsindo kusuka kumakrofoni eyakhelwe ngaphakathi kuya kuphayiphi yangemuva: I-Whisper yokubhala umbhalo, i-GPT‑4o yokukhiqiza izimpendulo zesitayela se-Wheatley kanye ne-ElevenLabs yokukhiqiza izwi elidumile. Umsindo ubuya nge-WebRTC, kanti i-ESP32 iphatha ukudlala, iguqula ngempumelelo idivayisi ibe yipropu ekhulumayo, eqhutshwa abalingiswa.

Ngasohlangothini oluwusizo kakhulu, kunezinsiza eziningi zezwi ze-DIY ezisebenza nge-ESP32 ezisebenza njengehabhu yomsindo nokulawula ene-Node.js, i-LangChain kanye ne-OpenAI backend. Ukusetha okujwayelekile kufaka inkinobho yokuqala/ukuyeka ukulalela, ukusakaza umsindo nge-WebSockets kuya kuphayiphi yamafu, kanye nezimpendulo zomsindo zesikhathi sangempela ezithunyelwa emuva futhi zidlalwe kudivayisi. Izindawo zokugcina ezivulekile zivame ukufaka imidwebo egcwele yokuxhuma, i-firmware kanye nekhodi yeseva, okwenza la maphrojekthi akwazi ukuphinda akhiqizwe futhi afundise.

Lezi zibonelo zigcizelela iphuzu eliyinhloko: I-ESP32 ayiseyona nje "imojula ye-Wi‑Fi ene-GPIO". Ngokwakhiwa okufanele, iba yisisekelo sama-ejenti asebenzisanayo, aphilayo futhi aqaphela umongo ahlala ezweni elibonakalayo futhi akhulume, alalele futhi aphendule ngezindlela ezimangalisayo zabantu.

I-Voice AI ihlanganisa i-ESP32‑S3, i-Dify, i-Xiaozhi kanye ne-Home Assistant

Kwabathandi bezindlu ezihlakaniphile kanye nabahlanganisi, kukhona uhlelo oluthakazelisayo kakhulu olwakhiwe ngamadivayisi e-ESP32-S3 njenge-SenseCAP Watcher, i-backend ye-Xiaozhi ESP32 kanye neplatifomu ye-Dify AI. Lesi sitaki siguqula i-Watcher ibe yi-interface yezwi engenazandla ye-Home Assistant, ene-ejenti ye-AI engaqonda umongo, ibuze izimo zedivayisi futhi ikhiphe imiyalo ngamathuluzi e-MCP.

Ukwakheka okuphelele kubukeka kanje: I-Dify isebenza njenge-"brain" ye-AI, iseva ye-Xiaozhi ESP32 ihlanganisa ihadiwe kanye ne-AI, kanti i-SenseCAP Watcher inikeza isikhombimsebenzisi somuntu. I-Dify isingatha uhlelo lokusebenza lohlobo lwe-Agent oluxhunywe kumhlinzeki we-LLM (i-OpenAI, i-Azure OpenAI, i-Volcano Engine, i-MiniMax, njll.), kuyilapho i-Xiaozhi ithola izingxenye zomsindo ezivela ku-ESP32, yenza ukuqashelwa kwenkulumo futhi ithumele umbhalo ovelayo ku-ejenti ye-Dify.

Ngasohlangothini lwe-Dify, ulungiselela okungenani umhlinzeki wemodeli oyedwa kuzilungiselelo zesikhulumi, bese udala uhlelo lokusebenza lwe-Agent olusebenza njengomphathi wakho ohlakaniphile. Ukhiqiza ukhiye we-API wohlelo lokusebenza, osetshenziswa yi-Xiaozhi ukuze ikwazi ukudlulisa amazwi abasebenzisi kuhlelo lokusebenza olufanele lwe-Dify futhi ithole izimpendulo. Lokhu kuhlanganisa yonke ipayipi ngaphandle kwezimfihlo zokubhala ikhodi ku-firmware ye-microcontroller.

I-backend ye-Xiaozhi ngokwayo ivame ukusebenza ku-Docker isebenzisa ukuthunyelwa kwe-module ephelele. Ngemva kokufaka, ulungiselela amapharamitha afana server.secret nama-URL angaphandle, qinisekisa ukuthi isitsha se-Xiaozhi singafinyelela isitsha se-Dify API ngenethiwekhi ye-Docker (ngokuvamile ku- http://dify-api-1:5001/v1), bese uqala kabusha ukuze usebenzise ukucushwa. Ikhonsoli inikeza i-UI yewebhu echwebeni elifana ne-8002, lapho uphatha khona ama-ejenti namadivayisi.

Ekugcineni, ubhalisa i-SenseCAP Watcher ne-Xiaozhi ngokulungisa ikheli leseva le-OTA ku-portal ebanjwe yedivayisi (isibonelo, 192.168.101.109:8002), ukuyivumela ukuthi iqalise kabusha futhi ifunde ikhodi yokuqinisekisa, bese ifaka leyo khodi esikrinini sokuphathwa kwedivayisi ye-Xiaozhi. Kusukela ngaleso sikhathi kuqhubeke, i-Watcher ingacela izibuyekezo ze-OTA, ivule ukuxhumana kwe-WebSocket futhi ihlanganyele ngokugcwele emsebenzini wokusiza ngezwi.

Ukuxhumanisa ama-ejenti e-Dify ku-Home Assistant ngamathuluzi e-MCP

Ukuze wenze i-ejenti ye-Dify ilawule ngempela amadivayisi ahlakaniphile asekhaya, uyawandisa ngethuluzi elisekelwe ku-MCP elikhuluma nomsizi wasekhaya. Esigabeni sikaDify esithi “Amathuluzi”, uthola i-plugin ye-MCP SSE, uyifake bese unikeza ukucushwa kwe-JSON okuchaza indlela yokufinyelela i-instance yakho ye-Home Assistant kanye nokuqinisekisa.

Lokhu kulungiselelwa kuvame ukufaka i-URL ekhomba iseva ye-MCP yoMsizi Wasekhaya kanye nethokheni yokufinyelela ehlala isikhathi eside. Ukhiqiza ithokheni kuphrofayela yomsebenzisi we-Home Assistant ngaphansi kokuthi “Amathokheni Okufinyelela Ahlala Isikhathi Eside”, bese uyifaka ku-JSON eceleni kwe-URL efanele ye-SSE, ngokuvamile into efana naleyo http://YOUR_HA_IP:8123/api/mcp kuye ngokuthi iseva ye-MCP isethwe kanjani.

Uma isilondoloziwe, i-Dify iqinisekisa ukucushwa kwe-MCP futhi iveze ithuluzi le-Home Assistant ku-ejenti yakho. Ukusuka lapho, i-prompt yakho iba yisihluthulelo: esigabeni se-prompt se-Agent uchaza indima yaso, uchaze ukuthi singashayela ithuluzi le-MCP ukuvula nokuvala amadivayisi, ukufunda izimo ze-sensor, njalo njalo, bese siliyala ukuba libuze imibuzo ecacisayo lapho imiyalo ingacacile.

Ngesikhathi sokusebenza, ukuhamba komsebenzi kuzwakala kungokwemvelo: ukhuluma ne-SenseCAP Watcher, uXiaozhi uguqula umsindo ube umbhalo, i-ejenti kaDify ihumusha isicelo futhi, uma kudingeka, ishayela ithuluzi le-MCP ukuze lisebenzisane ne-Home Assistant. Izenzo nezimpendulo zedivayisi eziphumayo zihunyushwa zibe yimpendulo ekhulunywayo yomsebenzisi, zakha iluphu ephelele yengxoxo eqhutshwa yi-ejenti ye-AI kodwa ehlanganiswe ngokujulile nesistimu yekhaya elihlakaniphile lendawo.

Lo mklamo ugcina i-logic ye-AI enzima ku-Dify ngenkathi uvumela i-backend ye-ESP32 S3 kanye ne-Xiaozhi ukuthi zigxile ekuphatheni umsindo we-low-latency kanye nokuphathwa kwamadivayisi okuphephile. Kuyisibonelo esihle sendlela ifu kanye nomphetho okungahambisana ngayo esikhundleni sokuncintisana, ikakhulukazi ezimweni eziyinkimbinkimbi zokwenza izinto ngokuzenzakalela ekhaya.

Izingxoxo ze-OpenAI Realtime, i-ElatoAI kanye nefomu elide ku-ESP32 S3

Okunye ukujikeleza kwesimanje kuma-ejenti e-AI asekelwe ku-ESP32 kuvela ekuqalisweni kwereferensi ye-ElatoAI kusetshenziswa i-OpenAI's Realtime API. Umgomo walapho ukusekela izingxoxo zenkulumo-kuya-enkulumweni ezingaphazanyiswa zemizuzu engaphezu kweyishumi, kusetshenziswa i-ESP32-S3, i-Secure WebSockets kanye ne-Deno Edge Functions ukuze kube nokubambezeleka okuphansi emhlabeni jikelele.

I-ElatoAI ihlelwe ngezingxenye ezintathu eziyinhloko: i-frontend ye-Next.js (evame ukusetshenziswa ku-Vercel) yokuphatha izinhlamvu ze-AI nokukhuluma nazo kusuka kusiphequluli, imisebenzi ye-edge esekelwe ku-Deno yokusingatha ukuxhumana kwe-WebSocket kanye nezingcingo ze-OpenAI, kanye neklayenti le-ESP32 Arduino elisakaza umsindo uye kuseva ye-edge futhi usuka kuyo. I-Supabase inikeza ubuqiniso, ukuphathwa kwedivayisi kanye nesitoreji semibhalo yengxoxo kanye nedatha yokucushwa.

Iresiphi yehadiwe incane ngamabomu: ibhodi lokuthuthukisa le-ESP32‑S3, imakrofoni ye-I2S efana ne-INMP441, i-amplifier ye-I2S efana ne-MAX98357A enesikhulumi esincane, inkinobho yokucindezela noma inzwa yokuthinta yokusebenzisana kanye ne-RGB LED yempendulo ebonakalayo. Akukho PSRAM edingekayo ngokuphelele ngenxa yokusebenzisa kahle ukucindezelwa komsindo we-Opus nokusakaza; lokhu kugcina umsindo wezinto uphansi ngenkathi kunikeza ikhwalithi yezwi ehlanzekile.

Ngasohlangothini lwenethiwekhi, i-ESP32 ivula i-portal evalelwe ukuze umsebenzisi akwazi ukumisa iziqinisekiso ze-Wi‑Fi, bese exhuma kabusha futhi abhalise idivayisi nge-Supabase esebenzisa ikheli layo le-MAC kanye nekhodi echazwe ngumsebenzisi. I-firmware ixhuma kuseva yomphetho we-Deno kanye ne-frontend ye-Next.js ekhonjwe ama-IP endawo athuthukiswayo noma izizinda ezifanelekile ngokugcwele ekukhiqizweni, kuzo zonke izixhumanisi ze-WSS eziphephile.

Ngokombono womsebenzisi, i-ElatoAI ikuvumela ukuthi ukhethe phakathi kwezinhlamvu ze-AI ezahlukene, udale ubuntu obungokwezifiso bese ubacindezela kudivayisi ye-ESP32. Ivolumu ingalawulwa kusuka kuhlelo lokusebenza lwewebhu, i-firmware ingabuyekezwa emoyeni, futhi imibhalo igcinwa ku-Supabase ukuze ibuyekezwe kamuva. I-WebRTC isetshenziselwa ukusekela izingxoxo ezingaphakathi kwesiphequluli, kuyilapho i-WebSockets isingatha ukuxhumana kwedivayisi, inikeze ulwazi oluqhubekayo lwe-multi-endpoint.

Lapho ama-ejenti endawo e-ESP32 ekhanya khona: amacala okusetshenziswa abalulekile

Uma usuvumile ukuthi i-ESP32 ayikwazi ukusingatha amamodeli amancane kuphela kodwa nama-loop agcwele ejenti, izinhlelo zokusebenza eziningi zomhlaba wangempela ziyavuleka. Ekwenzeni okuzenzakalelayo kwasekhaya, amanxusa endawo angafunda amaphethini okusebenzisa, afiphaze noma akhanyise izibani ngokusekelwe ekubeni khona nesikhathi sosuku, noma ashukumise i-thermostat ngobuhlakani ngaphandle kokufaka i-spam efwini ngokufundwa ngakunye kokushisa.

Kwezolimo kanye ne-IoT yasemakhaya, lapho i-bandwidth ingaba yindlala futhi ibiza kakhulu, ama-ejenti e-ESP32 angenza izinqumo mayelana nokunisela, ukungenisa umoya noma amafasitela okugcina ukushisa ngokusekelwe ezinzwa zesimo sezulu sendawo kanye nedatha yomlando. Izibalo ezihlanganisiwe noma izexwayiso ezibalulekile kuphela okudingeka zibuyele kuseva ephakathi, okunciphisa kakhulu izindleko zedatha nokwenza uhlelo luqine kumanethiwekhi angahlelekile.

Izindawo zezimboni ezilula zingenye indawo enhle. Amabhodi e-ESP32 ahlonyiswe ngama-accelerometer kanye nezinzwa zokushisa angasebenza njengama-node okulungisa abikezelayo, asebenzise amamodeli amancane okuthola okungajwayelekile endaweni ukuze abonise ukudlidliza okungavamile noma ukushisa ngokweqile futhi aqalise izexwayiso zangaphambi kwesikhathi ngaphambi kokuba imishini ihluleke. Ngenxa yokuthi ukuphetha kusebenza kudivayisi, uhlelo luyaqhubeka nokusebenza noma ngabe ukuxhumana kwehla ngesikhathi sewindi lokukhiqiza elibalulekile.

Imfundo kanye namarobhothi kuyazuza futhi kulezi zinhlaka zama-ejenti. Isibonelo, nge-PycoClaw, izikole zingakha amarobhothi ashibhile noma ukufakwa okusebenzisanayo lapho ukuziphatha kungagcini nje ngokuba yikhodi eqinile kodwa kuvumelana nezimo, ngenkumbulo eyisisekelo yokuxhumana kanye ne-interface yezwi elula. Ihadiwe ishibhile ngokwanele ukuthi wonke amakilasi angaba nokufinyelela okusebenzayo.

Ezimweni zokuthengisa noma zomphakathi, abasizi abasebenzisa i-ESP32 bangasebenza njengezindawo zokusebenzela, izindawo zolwazi noma abasizi bokufinyelela. Bangabingelela izivakashi, banikeze imiyalelo ekhulunywayo, baphendule ezinzwa (njengokunyakaza noma ukusondela) futhi baqhubeke nokusebenza bengaxhunyiwe ku-inthanethi, idatha ebucayi ingaphumi endaweni ngaphandle uma kudingeka ngokucacile.

Imikhawulo, izinselele kanye nalokho okufanele ukuqaphele

Naphezu kwazo zonke izimo zokusebenzisa ezithembisayo, ama-ejenti e-AI endawo ku-ESP32 anemingcele emikhulu okufanele uyihloniphe. Ukubala kanye nenkumbulo kuqinile, ngakho-ke noma yini engaphezu kwamamodeli amancane, agxile kumele idluliselwe kwinsizakalo yamafu. Uma uhlelo lwakho lokusebenza luncike ekucabangeni okucebile kolimi lwemvelo, cishe uzodinga i-LLM endaweni ethile.

Usayizi wemodeli ungenye yezingqinamba eziyinhloko: ezilungiselelweni eziningi une-flash engaphansi kwe-1 MB etholakalayo ye-AI, okwenza ukwakheka okucophelelayo kanye nokwenza ngcono kube yimfuneko engaxoxiswani. Cishe kuzodingeka uhlanganise ukulinganisa, ukucheba, ukunciphisa ungqimba kanye nokuhlela ngobuhlakani ukuze izinto zihambe kahle ngaphandle kokuphahlazeka ngenxa yezimo zokungasebenzi kahle kwenkumbulo.

Ukubuyekeza ama-ejenti namamodeli ngezinga elikhulu kungenye inkinga engeyona into encane. Nakuba izinhlelo ezifana ne-PycoClaw zivumela ukulungisa ubuntu be-ejenti kanye nemithetho ngamafayela ombhalo ahlelekayo, ukufaka esikhundleni imodeli eyisisekelo kumadivayisi amaningi noma amakhulu kusadinga ipayipi le-OTA eliqinile kanye nokuhlanzeka okuhle kokusebenza, ikakhulukazi lapho ukuxhumana kuqhubeka ngezikhathi ezithile noma amadivayisi esetshenziswa ezindaweni ezinzima.

Ezokuphepha zidinga ukunakwa okukhethekile ngokushesha nje lapho amanxusa akho efinyelela kunoma yini eyigugu noma engaba yingozi. Izici ezifana ne-secure boot, i-encrypted flash, i-signed firmware, i-mutual TLS, ukugunyazwa okusekelwe ezindimeni kanye nokubhalisa okuphelele akuzona izinto ozikhethela zona ezimweni zezimboni. Ngenxa yokuthi ama-AI agents angase asebenzise amathuluzi futhi asebenzise i-dynamic logic, kufanele ucacise kakhulu ngalokho angakwenza nalokho angenakukwenza.

Okokugcina, ezinye zezindawo zemvelo ezithuthuke kakhulu zisesencane kakhulu. I-PycoClaw, i-ScriptoHub kanye namaphethini athile okuhlanganisa e-Xiaozhi/Dify ashintsha ngokushesha; imibhalo ingase ingasebenzi kahle ezicini ezintsha futhi abasebenzisa izinto ezintsha kumele bazizwe bekhululekile ukusebenza nama-API asheshayo kanye namathuluzi aqhutshwa ngumphakathi. Ngenxa yalokho, uthola ukufinyelela kusenesikhathi kumakhono angahlukanisa umkhiqizo wakho ngaphambi kokuba yonke imakethe ifinyelele.

Uma sihlanganisa konke, isithombe esivelayo siwukuthi i-ESP32 iphuma “kumojuli ye-Wi‑Fi eshibhile” yaba isisekelo sama-edge node ahlakaniphe ngempela, akwazi ukubona, ukukhumbula, ukucabanga (endaweni noma ngefu) nokwenza izinto ezweni elingokoqobo. Ngezinhlaka ezifana ne-ESP Claw kanye ne-PycoClaw, ama-hybrid voice stacks asebenzisa i-LangChain, i-MCP noma i-OpenAI Realtime, kanye nezibonelo zomhlaba wangempela njenge-cyberpets, ama-Wheatley replicas kanye nama-Home Assistant butlers, ama-AI agent endawo ku-ESP32 asevele ewusizo, anamandla futhi alungele ukusekela igagasi elilandelayo le-IoT, amarobhothi kanye nemikhiqizo yemvelo ehlakaniphile.