Home | Community | Message Board

Magic Mushrooms Zamnesia
This site includes paid links. Please support our sponsors.


Welcome to the Shroomery Message Board! You are experiencing a small sample of what the site has to offer. Please login or register to post messages and view our exclusive members-only content. You'll gain access to additional forums, file attachments, board customizations, encrypted private messages, and much more!

Shop: MagicBag.co All-In-One Bags That Don't Suck, Certified Organic All-In-One Grow Bags   Myyco.com Golden Teacher Liquid Culture For Sale   North Spore North Spore Mushroom Grow Kits & Cultivation Supplies

Jump to first unread post Pages: 1
AI controlled pet robots
    #29336009 -

Hello everyone, post your own AI projects here, or anything AI related if you like.

this thread is mostly just a log and workspace, where i collect useful information.



i thought it might be fun to have a pet AI robot, and give it free-will to do whatever it wants,
also to answer questions and the usual chatbot things.. it maybe more complicated than i thought.

at first i thought, SO easy, get a 20$ voice-controlled toy robot, let the LLM tell it what todo.
(there are many different cheap toy robots with voice control, and App controlled, to choose from)
some cool but expensive pet AI robots you could buy

The problem is chatgpt deepseek etc can't see.. so it need to do some more research..



Quote:
Google:what LLM can see and navigate?"  a combination of different AI technologies working together
. The core LLM provides reasoning, but it is integrated into a larger system with specialized vision models and software that allows an agent to perceive and act within an environment, such as a website or a physical space.
Here is a breakdown of how these agents operate and some examples of systems:
How AI agents "see and navigate"

    Vision-Language Models (VLMs): Most standard LLMs are trained on text alone and cannot directly process images. To "see," an LLM is combined with a vision model, creating a VLM. The vision model analyzes visual input (e.g., images or video frames), and the VLM uses both the visual data and text to reason and answer questions.
    Navigation for a physical robot: In robotics, an LLM acts as the high-level planner and uses a multimodal VLM to interpret sensor data from its environment. This information is then translated into low-level instructions for the robot to move, grasp objects, or perform other physical actions.
 
Examples of LLM-based systems for navigation
Physical navigation

    NavGPT: This agent uses GPT models to perform high-level planning for visual navigation tasks. It breaks down complex instructions into sub-goals, integrates common-sense knowledge, and tracks navigation progress by processing textual descriptions of visual observations.
    VLMnav: An embodied AI framework from researchers at the University of Pennsylvania that uses a Vision-Language Model as an end-to-end navigation policy. It directly translates visual and language input into navigation actions without relying on separate modules for perception, planning, and control.
    LM-Nav: A system designed for robots that uses multiple pre-trained models. It employs GPT-3 as the LLM for high-level planning, CLIP as a VLM to connect visual data to landmarks, and a separate Vision-Action Model for physical control.
    VELMA: An LLM-based embodied agent for urban navigation using Street View imagery. It verbalizes visual observations and combines them with human-generated instructions to determine its next action.





Quote:
AI Overview
Yes, different Large Language Models (LLMs) can work together through several collaborative strategies, such as using frameworks like LlamaIndex to compose models, employing LLM ensembles to combine multiple models for better performance, or using techniques like model merging and federated cooperation. This multi-LLM approach allows for specialized task handling, improved accuracy, and more efficient processing, similar to how specialized team members work on complex projects.
Methods for LLMs to Work Together

    Model Composition:
        A single application can orchestrate multiple LLMs, sending a task to different models or using one model's output as the input for another.

Frameworks like LlamaIndex help manage these interactions, allowing developers to mix and match models for different parts of a task.
This is useful for cost-efficiency (using a cheaper model for drafts and a powerful one for refinement) or for accuracy (using a specialized model for a complex query).

Ensemble and Merging:

    Ensemble methods: use multiple LLMs to handle user queries during inference, combining their strengths to achieve better overall performance, according to an arXiv survey on LLM ensembles.

Model merging: involves combining the parameters of different LLMs into a single model, aiming for a solution that incorporates the best aspects of each, notes a YouTube video.

Complementary Cooperation:

    Different LLMs can take on complementary roles; for example, one model could extract information while another validates its accuracy or checks against external sources.

This also involves pairing a small, faster model with a larger one for more efficient data processing, as seen in techniques like "Speculative Decoding".

Federated Cooperation:

    This approach, used for privacy-sensitive data, involves training or using local small models in conjunction with larger, cloud-based models.

Methods like Federated Prompt Engineering use local models to protect private data while leveraging powerful cloud LLMs for execution



Quote:
Multi-Agent Systems and Roles

    Expert Specialization:
    Each LLM in the system can be specialized for different tasks or areas of expertise, such as data analysis, creative text generation, fact-checking, or specific domain knowledge.

Hierarchical Structure:
A "boss" LLM or controller can break a large, complex task into smaller, manageable subtasks.
Agent Coordination:
These subtasks are then assigned to specialized "agent" LLMs, which execute their part of the plan.
Communication:
Agents can communicate and share information with each other to ensure that their individual tasks are coordinated and lead to a cohesive final output.

Frameworks for Collaboration

    Agent-Based Frameworks:
    This approach structures the system so that each LLM acts as an intelligent agent with a specific, defined role.

Co-LLM (Cooperative LLM):
This framework pairs a general-purpose LLM with specialized expert models. A "switch variable" acts like a project manager, using machine learning to decide when the general model needs input from the expert model for a specific word or token, leading to more accurate and efficient responses.
LangChain:
This open-source framework enables developers to combine different LLMs with other tools and systems to build complex, end-to-end AI applications by chaining them together.

Benefits of Multi-LLM Collaboration

    Increased Accuracy:
    By leveraging domain-specific expertise, the combined system can produce more accurate results, especially for complex or specialized prompts.

Enhanced Efficiency:
Specialized models can perform their tasks more efficiently than a single, large general-purpose model.
Improved Problem-Solving:
The "dream team" approach allows for a more comprehensive and robust solution by combining the unique strengths of various LLMs.
Mitigation of Limitations:
One LLM can compensate for the weaknesses or "blind spots" of another, such as fact-checking to prevent hallucinations or bias



Quote:
AI Overview
Agent-based frameworks for multi-LLMs are platforms that enable multiple, specialized Large Language Model (LLM) agents to collaborate on complex tasks by mimicking a team workflow. These frameworks provide the infrastructure for agents to communicate, coordinate, plan, use tools, and maintain memory, moving beyond single-LLM limitations to solve problems in a scalable and efficient manner. Key examples include LangChain, AutoGen, CrewAI, and MetaGPT.
How they work:

    Specialized Agents:
    Instead of one LLM doing everything, multiple agents are created, each with a distinct role or expertise (e.g., a planner agent, a researcher agent, a coding agent).

Collaboration and Communication:
Agents interact with each other to share information, discuss plans, and provide feedback, simulating a human team's collaborative process.
Tool Use and Memory:
Agents can integrate with external tools (like search engines or databases) and maintain memory of past interactions to better understand context and execute complex multi-step tasks.
Orchestration:
The framework manages the flow of information, task delegation, and decision-making among the agents, ensuring they work together effectively towards a common goal.

Key Components and Features:

    Memory Management:
    Agents need to store and recall past information to maintain context and make informed decisions.

Planning Capabilities:
Frameworks help agents generate detailed plans and break down complex tasks into smaller, manageable steps.
Tool Integration:
Agents can be equipped with functions to access external APIs, databases, and other software, expanding their capabilities beyond just text generation.
Communication Protocols:
Mechanisms are in place for agents to send and receive messages, share data, and coordinate their actions.
Workflow Automation:
These frameworks streamline the process of building complex AI-driven applications by providing plug-and-play modules and predefined architectures.

Popular Examples:

    LangChain/LangGraph:
    A popular framework that provides abstractions for building LLM-powered applications, with LangGraph enabling the definition of complex inter-agent interactions.

AutoGen:
Designed to build conversational multi-agent applications, allowing for flexible and customizable agent dialogues.
CrewAI:
Focuses on creating collaborative agent teams, where agents work together to complete tasks.
MetaGPT:
Another framework that supports team-based LLM agents for complex software development and task execution



i was thinking if you could have an AI (or group of AIs)
that could see, reason, and control an app (usually an android app)
it could bring to life any robot, drones, etc, anything that works with an app...

*so after searching threw almost ALL of the app controlled robots, and toy camera cars,
i found one that was a little higher price, but it seems to perfect for this project...
https://meborobot.com/features.php
(the one i found was very cheap as it is "used and untested", so i hope it works)

i had no idea, but another company already has similar project with the same bot i got...
Quote:
https://www.personalrobots.biz/nabot-ai-reinventing-robots-with-a-new-powerful-ai/

Key Features of the Nabot AI

    Autonomous actions:
    The robot can identify and pick up objects, and even mop floors.

Object recognition:
It uses an offline version of the TensorFlow AI Image Classifier to identify objects in its environment.
Mobile app integration:
It relies on a smartphone or tablet to run its neural networks for image processing.
Educational and STEM focus:
The Nabot AI is designed for STEM educators and aspiring engineers to learn about robotics and AI.

How it Works

    Hardware modification:
    Ximpatico took the Mebo 2.0 hardware and added their own AI and firmware to create Nabot AI.

AI processing on the phone:
All AI calculations and image processing are performed on a connected smartphone for security and performance.
Offline functionality:
Once the internet connection is used to log in to the app, Nabot can work offline

The "Nabot AI app" is the companion app for the Nabot AI robot, a trainable and programmable home robot designed to pick up toys, map objects, and even mop floors. The app, available for smartphones, allows users to control the robot, perform pre-programmed tasks with a few clicks, or code advanced functions using a simple, visual interface or a Python API for more complex projects. The app processes data on the smartphone to identify objects, and it also offers access to a community where users can share their custom programs. 
What the Nabot AI app does:

    Control the robot:
    Users can direct the robot to perform various tasks and functions.

Run pre-programmed tasks:
The app allows for easy execution of pre-written tasks, such as fetching or mopping.
Code and program the robot:
Users can create their own programs using a visual, block-based interface inspired by Google's Blockly.
Utilize a Python API:
For advanced programming, a Python Software Development Kit (SDK) provides flexibility for more complex tasks.
Process data offline:
The app uses an offline version of TensorFlow AI Image Classifier to identify objects, with all calculations and image processing done on the user's smartphone.
Connect with a community:
The app provides access to a growing library of user-shared programs and tasks.

How it works with the robot:

    The robot is equipped with a camera, wheels, and a prehensile arm to execute tasks.

The mobile app interfaces with the robot, allowing users to train it, program it, and control its actions.
The app provides a user-friendly way to interact with a real-world robot, teaching programming and STEM concepts.




Quote:
(A programmable API controlled by a Large Language Model (LLM) is an architecture where an AI agent, powered by the LLM, makes autonomous decisions on which API calls to execute based on natural language instructions
. The LLM acts as the orchestrator, interpreting a user's request and programming the right sequence of API calls to fulfill the task.
This differs from a conventional LLM API, where a user's application directly sends a request to the LLM and receives a text response. With a programmable API, the LLM is an intelligent, automated component within a larger workflow, capable of using tools (the APIs) to perform real-world actions)




https://github.com/crlane/python-mebo <<<a python based API works on MEBOv.1 but untested on MEBOv.2

How I’d Teach a 10-Year Old to Build AI Agents (n8n, No Code)

This mini GPU runs LLM that controls this robot

"This mini GPU runs LLM that controls this robot"

2free offline AI-agent builders
https://www.sim.ai/
https://n8n.io
and more...
Quote:
google: what are some free and offline n8n alternatives....

1. Activepieces
Activepieces is an open-source workflow automation tool that is a simpler alternative to n8n, with a cleaner user interface.

    Best for: Individuals and teams seeking a no-code, Zapier-like experience in a self-hosted environment.
    How it works: Like n8n, it uses a visual, drag-and-drop interface for building workflows.
    Key features:
        Self-hosting capability to maintain control over your data.
        A growing library of integrations.
        AI-powered features for generating workflows.
        Designed to be more accessible for non-technical users than n8n.

2. Node-RED
Originally developed by IBM, Node-RED is a flow-based programming tool particularly popular for Internet of Things (IoT) applications.

    Best for: Technical users and developers integrating hardware, APIs, and custom web services.
    How it works: You create flows by visually wiring together "nodes" in a web-based browser editor.
    Key features:
        Completely free and open-source.
        Can be deployed on a local machine, private cloud, or edge devices like a Raspberry Pi.
        Extensible node ecosystem for MQTT, HTTP, and databases.
        Supports custom JavaScript function nodes for complex logic.

3. Huginn
Dubbed a self-hosted, "hackable" version of IFTTT or Zapier, Huginn allows you to build agents that perform automated tasks for you.

    Best for: Users who want maximum customization and privacy for online tasks like web scraping and event monitoring.
    How it works: You build complex conditional logic by configuring autonomous "agents" that watch for events, fetch data, and take actions.
    Key features:
        Runs on your own server, so all your data remains private.
        Uses a simple web interface for managing workflows.
        Highly extensible, allowing you to create custom agents.
        An older platform with a dated UI and a steeper learning curve than newer alternatives.

4. Automatisch
Automatisch is another self-hosted, open-source workflow tool that focuses on ease of use with a no-code interface.

    Best for: Non-developers who want a straightforward, self-hosted Zapier alternative for automating business tasks.
    How it works: You build flows with triggers and actions, connecting to various services through their APIs.
    Key features:
        Intuitive, drag-and-drop interface.
        Full data sovereignty by running on your own infrastructure.
        Actively developed, with contributions from a community of users.







Edited by ellomello (09/06/25 07:12 PM)

Extras: Filter Print Post Top
Re: AI controlled pet robots [Re: ellomello]
    #29337939 -

IDEAL GOALS for PROJECT MEBO3.0 APP
-an "AI agent" chatbot with vision (LLM+VLM) controls all functions of the mebo robot app in  autonomous mode: LLM drives around, explores, learns, comments, interacts, can use the arm, speak threw speakers, hear threw the microphone, take photos and videos when asked, or if LLM wants to. .
-app can work offline with a local LMM or in a human driver mode.
-app can work online with any LLM, or a remote human connected over the internet to the app.
-app works on android tablet or phone, (a windows version would be nice too)
-app should be easy to use with all controls: video/sound/speaker, wheels, arm, etc.
-app should have files for logs and photo/video
-human users online could have a log in account and each have their own files.
-online users could be able to connect to other mebo robots around the world, who give permission.
-*online users could choose to make their mebos connection public, or keep it private.
-**and choose if a their files folder is private or public.
-using a traffic monitoring tool (attached to the LLM/VLM by the AI agent) to notify of any unauthorized connection to it's network, the LLM could notify user when any new networks appear near by or connect to the mebo. Any other apps, nearby sensors, or websites could be tied in with the AI agent to send notifications.
-a small laptop, tablet, phone, or raspberrypi with a screen, could be attached onto the mebo, and the AI agent could connect it into the 'mebo system' so the LLM could send relevant images, videos or information, and have an animated face, and maybe it find clips to express reactions.
-it would be cool if LLM sometimes moves it's arm in expressive gestures while talking.
and why not....
-this app could also control any other app controlled devices, by user giving permission to use that app.
(visual-LLM to be able to control buttons on any apps, so it can work on anything controlled by any app)

someone make this!^! i tried to get sim.ai to build it, but it froze every time it was close to finishing.
*i mean it did turn my crazy words into some kind of program, but i ran out of credits, so who knows if it would have worked. i feel like the chat-coder was just telling me what i want to hear, and not really being honest.. anything you ask it would be "oh no problem let me just do this this and this, your all set!"
also it's expensive once you run out of credits, so idk if it would have worked, fun, but idk if it works.

picture of the old APP that doesn't even exist anymore, (or maybe it's only on apple store, f^>k apple.)


found some mebo python scripts.. thanks nerds =D

https://github.com/meborobot/letsrobot-mebo
https://python-mebo.readthedocs.io/en/latest/
https://github.com/crlane/mebo-hacking
https://github.com/crlane/python-mebo
https://python-mebo.readthedocs.io/en/latest/_modules/mebo/robot.html
https://github.com/csev1755/python-mebo2-nabot/tree/main

just more research on google::
Quote:
Mebo have a simple HTTP API that can be accessed to send specific commands and control functions.
The Mebo robot (version 1) is controlled by a simple, unauthenticated HTTP API
. The robot runs a web server that accepts commands as URL request parameters.
Mebo API details

    Base URL: All requests are sent to the Mebo's IP address on port 80.
    Command structure: A command and its parameters are specified in the query string of a GET request.
    Security: There is no authentication or encryption, so requests are made "in the clear".

Python-mebo library
For controlling a Mebo robot with code, a community-made Python library called python-mebo is available on GitHub. This is not an official tool from the manufacturer, Skyrocket LLC.

    Installation: pip install mebo.
    Usage: The library abstracts the HTTP requests, making control functions more user-friendly. The following examples show how to use the library to control the robot:
        Initialize the robot object:
        python

        from mebo import Mebo
        m = Mebo() # Auto-discovers the robot's IP

        Use code with caution.

Move forward at max speed for 1 second:
python

m.move('n', speed=255, dur=1000)

Use code with caution.
Move the arm up for 1 second:
python

m.arm.up(dur=1000)

Use code with caution.
Open the claw for 1 second:
python

m.claw.open(dur=1000)

Use code with caution.


API command structure
Details for the direct HTTP requests can be found in the mebo-hacking GitHub repository. The general format for commands is http://<mebo_ip>/command?param1=value1¶m2=value2.
Here are some example commands:

    Movement:
        Forward: /command?name=wheel_move&side=n&speed=255&duration=1000
        Backward: /command?name=wheel_move&side=p&speed=255&duration=1000
        Left turn: /command?name=turn&side=p&speed=255&duration=1000
        Right turn: /command?name=turn&side=n&speed=255&duration=1000
    Arm control:
        Arm up: /command?name=arm_move&side=n&speed=255&duration=1000
        Arm down: /command?name=arm_move&side=p&speed=255&duration=1000
    Claw control:
        Claw open: /command?name=claw_move&side=n&speed=255&duration=1000
        Claw close: /command?name=claw_move&side=p&speed=255&duration=1000

AI responses may include mistakes. Learn more

    crlane/mebo-hacking: Description of the mebo api - GitHub
    Hacking the Mebo. Mebo v1. The Mebo toy robot via controlled by a simple http API. Requests are made to a web server on the Mebo at port 80. They are made in th...
    favicon
    GitHub

python-mebo's documentation! — python-mebo 0.1.0.b1 ...
Mebo is a python package to control the Mebo Robot with Python. It is currently in active development, so there might be breaking changes. Some basic usage is d...
favicon
Read the Docs
A simple python library to control the mebo toy robot - GitHub
Quickstart. Some basic usage is below. The API will change and limited documentation exists, but it works for getting started. from mebo import Mebo m = Mebo() ...
favicon
GitHub

Show all
can mebo be detected and connected like normal wifi connection
Yes, a Mebo robot can be detected and connected to like a normal Wi-Fi device
. When you turn on Mebo, it broadcasts its own wireless network, which your phone, tablet, or computer can connect to directly. This is known as a direct hotspot connection.
The setup process generally works in one of two ways:

    Direct hotspot: In its default mode, Mebo's built-in Wi-Fi creates a temporary network that you can see and connect to from your device's Wi-Fi settings. This allows you to control the robot without needing an internet connection, but the range is limited.
    Home Wi-Fi network: For extended range, you can use the official app to configure Mebo to join your home's Wi-Fi network. Once it is connected to your local network, any device on the same network can control it.

How to connect to Mebo

    Turn Mebo on using the power switch.
    Go to the Wi-Fi settings on your phone, tablet, or computer.
    Look for a network named something like MEBO2-xx-xx-xx and connect to it.
    Open the Mebo app (or use the web API) to establish the control link. You should hear a confirmation sound from the robot.

If you have trouble connecting or the robot is unresponsive, you can press the reset button on its underside to clear the Wi-Fi settings.
AI responses may include mistakes. Learn more

    Support - Mebo Robot
    FAQs * How do I pair my phone with Mebo? ▼ First, download the free app available in the Apple App Store or the Google Play store. Next, turn on Mebo and open y...
    favicon
    meborobot.com

MEBO ROBOT, LIPO BATTERY & CHARGER AGES 8+
* 2. CHARGING MEBO's BATTERY. • Plug battery into USB Charger. • Plug USB Charger into USB port (Not included). • Charge until battery is completely charged. (L...
favicon
meborobot.com
Mebo ROBOT, LIPO BATTERY & USB CHARGER
MEBO CONTROLS. 1. INSTALLING MEBO'S BATTERY • Open the battery compartment on the back of Mebo. ... Plug the battery into the enclosed connector. Rest the batte...
favicon
images-na.ssl-images-amazon.com

Show all
once connected, can the python commands can be sent to mebo via the browser search bar
Yes, once you are connected to the Mebo's Wi-Fi network, you can control it by sending HTTP requests directly from a web browser's address bar
. However, you cannot use Python commands, because the browser cannot interpret Python code. Instead, you must use the specific HTTP API commands that the Mebo robot understands.
How to send commands via browser
The browser's address bar works by sending a GET request to the URL you enter. You can craft a URL with the Mebo's IP address and the correct API parameters to control the robot.

    Find the Mebo's IP address. When you connect your computer or phone to the Mebo's direct Wi-Fi network, you can find the assigned IP address in your network settings. If you configured the robot to join your home network, you can find its IP address from your router's administration page.
    Enter the command URL. A command URL will look like http://[Mebo's IP address]/command?[parameters].

Example:
To make Mebo move forward at max speed for 1 second, you would type this into your browser's address bar:
http://[Mebo's IP address]/command?name=wheel_move&side=n&speed=255&duration=1000
Limitations

    No Python code: You cannot type Python function calls like m.move('n') into the browser bar. The browser only understands standard URLs and HTTP requests.
    Basic commands only: The browser can only send simple, single commands. For more complex sequences of actions, using a Python script is much more effective, as it can chain multiple commands together.
    No feedback: Unlike a Python script that can receive and process the server's response, a browser will simply display the Mebo's response, which is often just a simple "OK" message.

AI responses may include mistakes. Learn more

    crlane/mebo-hacking: Description of the mebo api - GitHub
    Mebo v1. The Mebo toy robot via controlled by a simple http API. Requests are made to a web server on the Mebo at port 80. They are made in the clear with no au...
    favicon
    GitHub

How to make basic REST API calls using a browser - Stack Overflow
Mar 17, 2015 — Straight from the browser bar you can utilize REST endpoints that respond to a GET message. That is what you are doing when you hit that URI, you are sending an...
favicon
Stack Overflow
What HTTP request is made by browser when accessing endpoint ...
Apr 7, 2020 — * 3 Answers. Sorted by: 3. When you type a URL in the address bar of your browser, it performs a GET request to retrive the content at the specified end-point. ...
favicon
Stack Overflow

Show all
what other ways can i access port 80 to send the commands
Thinking
Kicking off 8 searches
Looking at 178 sites
Putting it all together
Besides using a browser's address bar, you can send commands to the Mebo robot's port 80 using
command-line tools or programming scripts. These methods offer more control, allow for automation, and are necessary for creating more complex control sequences.
Command-line tools
You can use a terminal or command prompt to send HTTP GET requests directly. This is useful for quick testing and scripting without needing to write a full program.
cURL
cURL is a widely available command-line tool for making network requests.

    Command: curl "http://[Mebo's IP]/command?name=wheel_move&side=n&speed=255"
    Availability:
        macOS and Linux: Pre-installed.
        Windows: Available in modern versions of PowerShell and Command Prompt.

PowerShell
On Windows, you can use the Invoke-WebRequest cmdlet.

    Command: Invoke-WebRequest -Uri "http://[Mebo's IP]/command?name=claw_move&side=n&duration=1000"

Programming scripts
For complex or automated behavior, writing a script in a language like Python or JavaScript is the best approach.
Python
The requests library is the standard way to make HTTP requests in Python.
Example script:
python

import requests
import time

mebo_ip = "192.168.x.x"  # Replace with your Mebo's IP address

# Send command to move forward for 1 second
def move_forward():
    url = f"http://{mebo_ip}/command?name=wheel_move&side=n&speed=255&duration=1000"
    requests.get(url)

# Send command to open the claw
def open_claw():
    url = f"http://{mebo_ip}/command?name=claw_move&side=n&duration=1000"
    requests.get(url)

# Move forward, wait, then open the claw
move_forward()
time.sleep(1)
open_claw()

Use code with caution.
JavaScript (Node.js)
If you are working with a Node.js environment, you can use the built-in fetch API.
Example script:
javascript

const meboIp = "192.168.x.x"; // Replace with your Mebo's IP address

// Send command to move forward for 1 second
function moveForward() {
    const url = `http://${meboIp}/command?name=wheel_move&side=n&speed=255&duration=1000`;
    fetch(url);
}

// Send command to open the claw
function openClaw() {
    const url = `http://${meboIp}/command?name=claw_move&side=n&duration=1000`;
    fetch(url);
}

// Move forward, wait, then open the claw
moveForward();
setTimeout(openClaw, 1000);

Use code with caution.
Mobile applications
The official Mebo app uses the same HTTP API to send commands, but you could also create your own custom mobile app for more specific functionality.

 

You would develop an application for iOS or Android that can connect to the robot's Wi-Fi network and send the necessary HTTP GET requests. This is a more advanced option that requires software development skills

...
how can i receive the streaming audio and video on the connected device
Thinking
Kicking off 8 searches
Looking at 114 sites
Putting it all together
Accessing the video and audio stream from a Mebo 2.0 without the official app is possible, but it requires reverse-engineering the robot's network protocol. There is no official, documented method for this. The Mebo 2.0 likely streams its video and audio over a specific IP address and port, possibly using a standard format like MJPEG or MPEG-TS.

What is likely happening

    When you connect your device to the Mebo's Wi-Fi network, the official app communicates with a specific IP address and port to retrieve the stream.
    The video stream itself is likely Motion JPEG (MJPEG). Many internet protocol (IP) cameras use this format, which is a sequence of JPEG images sent over an HTTP connection.
    The audio is probably streamed separately over a different port or in a combined MPEG-TS format.

How to receive the stream (advanced)
This is not an officially supported method and requires technical knowledge of networking and streaming protocols.
Step 1: Find the Mebo's streaming URL
You need to find the specific URL the Mebo uses to broadcast its stream. This would involve:

    Connect your computer to the Mebo's Wi-Fi network.
    Run a network traffic analysis tool, such as Wireshark, while the official Mebo app is streaming.
    Analyze the network traffic to find the IP address and port where the video and audio data are being sent. For an MJPEG stream, the URL might look similar to this:
    http://[Mebo's IP address]/video.mjpeg

Step 2: Use an appropriate tool to view the stream
Once you have the stream URL, you can use various free and open-source tools to receive and display it.
VLC media player
VLC is a versatile and widely used media player that can open network streams.

    Open VLC.
    Go to Media > Open Network Stream.
    Enter the full stream URL you discovered in Step 1 (e.g., http://192.168.1.xxx/video.mjpeg).
    Click Play.

FFmpeg command-line tool
FFmpeg is a tool for handling multimedia data. It can be used to view or save the stream to a file.

    To view the stream:ffplay http://[Mebo's IP address]/video.mjpeg
    To save the stream to a file:ffmpeg -i http://[Mebo's IP address]/video.mjpeg -t 10 output.mp4

Python script with OpenCV
If writing a custom Python script, the OpenCV library can capture and display the stream.
Example Python script:
python

import cv2

mebo_stream_url = "http://[Mebo's IP address]/video.mjpeg" # Replace with actual URL

cap = cv2.VideoCapture(mebo_stream_url)

if not cap.isOpened():
    print("Error: Could not open video stream.")
else:
    while True:
        ret, frame = cap.read()
        if not ret:
            break
       
        cv2.imshow('Mebo Stream', frame)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

cap.release()
cv2.destroyAllWindows()

Use code with caution.
The script displays the video feed in a window on your computer. Handling the audio stream is more complex and requires additional libraries and code.

...

how can i receive the streaming audio and video on the connected device
Accessing the video and audio stream from a Mebo 2.0 without the official app is possible, but it requires reverse-engineering the robot's network protocol. There is no official, documented method for this. The Mebo 2.0 likely streams its video and audio over a specific IP address and port, possibly using a standard format like MJPEG or MPEG-TS.

What is likely happening

    When you connect your device to the Mebo's Wi-Fi network, the official app communicates with a specific IP address and port to retrieve the stream.
    The video stream itself is likely Motion JPEG (MJPEG). Many internet protocol (IP) cameras use this format, which is a sequence of JPEG images sent over an HTTP connection.
    The audio is probably streamed separately over a different port or in a combined MPEG-TS format.

How to receive the stream (advanced)
This is not an officially supported method and requires technical knowledge of networking and streaming protocols.
Step 1: Find the Mebo's streaming URL
You need to find the specific URL the Mebo uses to broadcast its stream. This would involve:

    Connect your computer to the Mebo's Wi-Fi network.
    Run a network traffic analysis tool, such as Wireshark, while the official Mebo app is streaming.
    Analyze the network traffic to find the IP address and port where the video and audio data are being sent. For an MJPEG stream, the URL might look similar to this:
    http://[Mebo's IP address]/video.mjpeg

Step 2: Use an appropriate tool to view the stream
Once you have the stream URL, you can use various free and open-source tools to receive and display it.
VLC media player
VLC is a versatile and widely used media player that can open network streams.

    Open VLC.
    Go to Media > Open Network Stream.
    Enter the full stream URL you discovered in Step 1 (e.g., http://192.168.1.xxx/video.mjpeg).
    Click Play.

FFmpeg command-line tool
FFmpeg is a tool for handling multimedia data. It can be used to view or save the stream to a file.

    To view the stream:ffplay http://[Mebo's IP address]/video.mjpeg
    To save the stream to a file:ffmpeg -i http://[Mebo's IP address]/video.mjpeg -t 10 output.mp4

Python script with OpenCV
If writing a custom Python script, the OpenCV library can capture and display the stream.
Example Python script:
python

import cv2

mebo_stream_url = "http://[Mebo's IP address]/video.mjpeg" # Replace with actual URL

cap = cv2.VideoCapture(mebo_stream_url)

if not cap.isOpened():
    print("Error: Could not open video stream.")
else:
    while True:
        ret, frame = cap.read()
        if not ret:
            break
       
        cv2.imshow('Mebo Stream', frame)
        if cv2.waitKey(1) & 0xFF == ord('q'):
            break

cap.release()
cv2.destroyAllWindows()

Use code with caution.
The script displays the video feed in a window on your computer. Handling the audio stream is more complex and requires additional libraries and code.
AI responses may include mistakes. Learn more

    MJEPG stream from NVR using HTTP - Amcrest Forum
    Jan 18, 2025 — Re: MJEPG stream from NVR using HTTP. Post by jack7 » Thu Jul 21, 2022 4:14 am. Not all Amcrest camera support MJPG, and only with substream. Use camera Web UI ...
    favicon
    Amcrest

[All cameras] What is the HTTP URL to play the camera's MJPEG ...
Jun 9, 2025 — ANSWER. Please follow the steps below to configure the settings: Step 1. Modify Video Codec: Refer to the image below to modify the video codec to MJPEG(JPEG). ...
favicon
VIVOTEK Support Center
GStreamer In-Band Metadata for MPEG Transport Stream
Dec 28, 2024 — The client pipeline allows the H264 video decoding of the incoming stream to be displayed in the respective video while the metasink element dumps the metadata ...
favicon
RidgeRun

Show all
is there another way to find the ip address and port of streaming audio and video
Thinking
Kicking off 9 searches
Looking at 182 sites
Putting it all together
The video and audio stream from a Mebo 2.0 robot can be found by using a network traffic analysis tool. Because the manufacturer does not officially document this API, this method requires some technical knowledge of networking to find the specific IP address and port that the stream is broadcast on
.
Method 1: Using Wireshark (Desktop)
Wireshark is a powerful, free network protocol analyzer that can capture all network traffic passing through your computer's network interface.
Steps:

    Download and install Wireshark. Get the latest version from the official Wireshark website.
    Connect your computer to the Mebo's Wi-Fi network. This ensures that the traffic between your device and the robot is visible.
    Open Wireshark and start a capture. Select the wireless network interface that is connected to the Mebo.
    Start streaming on the Mebo app. Open the official Mebo app on another device (like a smartphone) and begin streaming video and audio. This will trigger the robot to send data over the network.
    Stop the Wireshark capture after about 30 seconds of streaming.
    Filter the traffic. To find the video and audio streams, apply filters in Wireshark. Since video streaming is often done over TCP, a filter like tcp.stream eq X is a good starting point, where X is a stream number. You can also sort the connections by the amount of data transferred to find the one with the most data, as this is likely the video stream.
    Examine the packets to find a URL ending in .mjpeg or similar, or look for RTSP packets. The packet data will reveal the full URL, including the IP address and port number.

Method 2: Using a port scanner like Nmap
Port scanning is a technique used to identify open ports and the services running on them. Nmap is a popular open-source tool for this purpose.
Steps:

    Download and install Nmap or its graphical front-end, Zenmap.
    Connect to the Mebo's Wi-Fi network.
    Find the Mebo's IP address. You can use ipconfig on Windows or ifconfig on Linux/macOS while connected to the Mebo's network to find the gateway IP address, which is often the robot's IP.
    Run a comprehensive port scan. For example, using the command line, run nmap -p- [Mebo's IP address]. The -p- flag scans all 65,535 ports to find all active services.
    Examine the results for open ports that might be associated with streaming protocols, such as rtsp (often port 554), or an unusual port that is not port 80.

Method 3: Using a network monitoring tool
Several software programs and mobile apps can scan a network and show which devices are connected, along with their open ports.
Tools:

    Fing (Mobile App): This free app for Android and iOS scans your network and lists all connected devices, their IP addresses, and open ports. Install it, connect to Mebo's network, and run a scan.
    Advanced IP Scanner (Windows): This utility is similar to Fing and can scan for devices and open ports on your network.





Basic summary what we know about mebo:
-Mebo has a hotspot that you can connect your network to, and it receive commands via port80, that can be accessed in browser tab, but easier to use http:\\ python code in the terminal
-it likely broadcast video on an unknown, but open URL:ipaddress, audio possibly on another URL, or the same.
-audio and video URL should be discoverable using a network monitoring tool, like the two listed above.
- once URL:IP is discovered you can freely connect to audio video..

other things we don't know
-still not sure about sending audio to the mebo speakers so the LLM or connected person can speak, or if this will be an issue.

-it might be easier to use something like rasberry pi, with a HAT?(connects pi to mebo)
*other projects have been successful with connecting LLMs and VLMs to toy robots with a pi.
but i think it would be better to use original hardware with newly designed super-app if possible.

-probably best to build the mebo control app separate from the LLM+VLM controlling app...
*make the AI-control app it's own program, so it can be used for controlling any app, including mebo*
HOW i think AI-controller-APP works; an AI agent connects the VLM and other programs to LLM...
a VLM translates what it sees into text so the LLM can understand what it 'sees'.
other programs connected by AIagent are also sending text info to the LLM, the LLM decides what todo.

How AI-control-app uses app controls, ..idk
...
Quote:
Local LLM-powered autonomous agents are AI systems that can independently execute complex tasks using a large language model (LLM) that runs directly on your personal hardware
. This setup offers significant benefits like enhanced privacy and the ability to function offline, as your data never leaves your device.
Key characteristics and components
Unlike traditional, cloud-based LLMs that simply respond to text, an autonomous agent can observe its environment, plan a multi-step solution, and take action to achieve a goal. The core components of these agents include:

    Large Language Model (LLM): The "brain" that provides natural language understanding and drives decision-making. Smaller, more efficient open-source LLMs like Mistral, LLaMA 3, and Phi-3 are compact enough for local deployment on consumer hardware.
    Reasoning and planning: The ability to break down high-level instructions into a sequence of smaller, manageable subgoals.
    Memory: Retains context from past interactions and stores relevant information to improve performance over time.
    Tool utilization: Can interact with local files, APIs, and other software to execute tasks and access real-time data.

Benefits of running agents locally
Using a local, autonomous agent provides several advantages over relying on cloud-based AI:

    Maximum privacy: Your data and conversations are processed and stored entirely on your own machine, without being sent to an external server.
    Offline functionality: The agent can operate in environments without an internet connection, which is crucial for remote or privacy-sensitive tasks.
    Cost-free operation: It eliminates the need for expensive API subscriptions and pay-per-token fees charged by commercial cloud providers.
    Full control: Users have complete control over the AI's behavior, configurations, and data.

Frameworks and tools for local agents
Several projects and tools help developers and enthusiasts create autonomous agents on local hardware:

    Ollama: A popular and user-friendly tool for running open-source LLMs like Llama 3, Mistral, and Phi-3 locally on Windows, macOS, and Linux.
    LocalAI/LocalAGI: A free and open-source alternative to OpenAI that is designed to run locally. LocalAGI is specifically its autonomous agent platform for building and deploying agents locally.
    AnythingLLM Desktop: An all-in-one desktop application that allows you to easily run an LLM locally with a one-click install. It's designed to be local-first, with all models, documents, and chats stored on your computer.
    Langflow: A tool that allows you to build local AI workflows and agents, with native integration for Ollama. It can be powered by GPUs from NVIDIA GeForce RTX and RTX PRO.
    Agentic/CrewAI/LangChain: Frameworks for building more complex agentic workflows. While simpler tools like Agentic and CrewAI are easier to start with, LangChain offers more power and customization for advanced users.

Example projects
Users and developers are already experimenting with local LLM agents for various projects:

    Personal voice assistants: A user created a GPT-like voice assistant for their Raspberry Pi using a small LLM like TinyLlama, a microphone, and a speaker.
    Smart home control: Others have integrated local LLMs with smart home systems like Home Assistant to enable voice-controlled operation of devices.
    AI research and data science: Autonomous agents are being used to automate complex data science workflows, from cleaning and preprocessing data to generating reports and recommendations. For those interested in building or experimenting with autonomous agents that run locally, several free and open-source frameworks and tools are available
. The agents generally require you to run your own Large Language Model (LLM) on local hardware and use the framework to build the agent's logic.
Agent frameworks
These frameworks provide the building blocks and infrastructure for creating custom autonomous agents.

    LocalAI/LocalAGI: LocalAI is a free, open-source platform that acts as a drop-in replacement for the OpenAI API, allowing you to run powerful language models locally. Its companion, LocalAGI, extends this with a platform for autonomous agents that run locally without needing to code.
    CrewAI: An open-source framework for orchestrating AI agents to work together collaboratively. You can define roles and goals for different agents, enabling them to complete complex tasks as a team.
    AutoGen: A framework by Microsoft for enabling multiple AI agents to converse with each other to accomplish tasks. It is free to use and allows for customizable, conversable agents that also support human participation.
    LangChain: A popular open-source tool for building LLM-powered agents. It provides all the components needed to create agents that can reason, plan, remember, and interact with tools or APIs.
    AgentGPT: This tool allows you to assemble, configure, and deploy autonomous AI agents directly in your browser. It was created with the vision of making the power of AI accessible to everyone.

Pre-built agents and platforms
These projects offer pre-configured agents for specific use cases, which you can run on your local machine.

    AutoGPT: A well-known open-source autonomous agent capable of performing tasks independently. Different versions and implementations are available, including some that do not rely on paid APIs.
    OpenDevin: A platform for autonomous software engineers powered by AI and LLMs. It is free, open-source, and can be run locally for end-to-end autonomous software development.
    GPT-Researcher: An autonomous agent that performs comprehensive online research on any topic and can generate detailed reports.
    Aider: An AI pair programmer that works directly in your terminal, acting as an autonomous coding agent.
    Goose: An on-device AI agent for handling entire software development projects, including writing, executing, and debugging code.

How to get started
To run a local LLM agent, you will generally need to:

    Select a framework: Choose a framework like CrewAI or AutoGen to define the behavior and collaboration of your agents.
    Get a local LLM: Download and run an open-source LLM, such as those in the Llama family by Meta, Mistral, or others. Tools like LocalAI can simplify this process.
    Install the necessary tools: Make sure you have the required software dependencies, such as Docker, Node.js, and Git.
    Assemble and configure your agent: Use your chosen framework to define the agent's roles, goals, and access to tools or local APIs



Edited by ellomello (09/08/25 02:21 PM)

Extras: Filter Print Post Top
Re: AI controlled pet robots [Re: ellomello]
    #29347816 -

Quote:
list all known api commands for the mebo 2.0  and any other known controls
Thinking
Kicking off 7 searches
Looking at 40 sites
Putting it all together
Known Mebo API commands are accessed via an unauthenticated HTTP server running on port 80
. The server listens for requests to specific endpoints, which often contain parameters for duration and speed.
The most comprehensive documentation comes from the crlane/mebo-hacking GitHub project and the python-mebo library, which reverse-engineered the API.
Found API endpoints and pages
The following endpoints and pages have been identified on the Mebo's web server at its IP address (typically 192.168.1.1):

    /status.html: Displays system status, including memory and baud rate.
    /system.html: Contains controls for Wi-Fi setup, resetting the video/controller board, and performing a factory reset.
    /upgrade.html: Shows firmware details and provides an interface for updating the firmware for both the video and controller boards.
    /nvram.html: A page with NVRAM parameters, though its full use has not been documented.

Identified API commands
The python-mebo library provides the most useful summary of the control commands by demonstrating how to use them with a Python script.
Movement commands

    m.move('n', speed=255, dur=1000): Moves the robot forward ('n') at a certain speed (speed) for a duration in milliseconds (dur).
    m.move('s', speed=255, dur=1000): Moves the robot backward ('s').
    m.move('e', speed=255, dur=1000): Rotates the robot right ('e').
    m.move('w', speed=255, dur=1000): Rotates the robot left ('w').
    m.arm.up(dur=1000): Moves the arm up for a duration.
    m.arm.down(dur=1000): Moves the arm down for a duration.
    m.arm.stop(): Stops arm movement.
    m.claw.open(dur=1000): Opens the claw for a duration.
    m.claw.close(dur=1000): Closes the claw for a duration.
    m.claw.stop(): Stops claw movement.

Camera controls

    Video Stream: The video stream is accessible via a URL. While the specific format is not provided, the python-mebo library connects to and uses the stream, indicating it can be accessed with the correct endpoint.

Other controls and hardware information

    Wi-Fi Connection: To control the Mebo, your device must be connected to its self-generated Wi-Fi hotspot, named "MEBO2 - xx-xx-xx".
    Reset Button: A physical reset button can be pressed to recalibrate motors or clear a forgotten Wi-Fi password. It is typically located on the underside of the robot.
    Hardware Modifications: The Mebo is a popular platform for modding. Hackers have replaced the internal controller board with a Raspberry Pi and a motor hat to gain full hardware control, bypassing the stock API entirely. This method provides the most granular control over individual motors.



Quote:
make a complete list of every python code that can work on the mebo 2.0 and including even theoretical untested code
While no one can provide a "complete list of every Python code," we can outline the existing and theoretical Python code that can control the Mebo 2.0. This is possible because the robot's functionality is accessible through a Wi-Fi HTTP API, which has been reverse-engineered and documented in open-source projects
.
The Mebo 2.0 was also used as the basis for the Nabot AI, which has its own Python API that can be adapted for the Mebo.
Existing Python libraries
These libraries have already reverse-engineered the robot's API and provide a simple Python interface to control its functions.
Library: python-nabot-mebo2 (Recommended)

    Source: A confirmed working version for the Mebo 2.0 is maintained on GitHub.
    Installation: pip install python-nabot-mebo2
    Key functionalities:
        Connection: Automatically discovers and connects to the Mebo 2.0 via its Wi-Fi hotspot.
        Movement: Sends commands to control the robot's six wheels (forward, backward, left, right, etc.).
        Arm and claw: Controls the robot's arm and opens/closes the claw.
        Camera and speaker: Captures the video stream and controls the audio output.
        Configuration: Allows for saving Wi-Fi network settings.

Library: python-mebo

    Source: An older library on PyPI, primarily for the Mebo 1.0, though some functions may work on the Mebo 2.0.
    Installation: pip install mebo
    Note: The developers acknowledge that it has only been tested on the Mebo 1.0, and the python-nabot-mebo2 library is the more reliable option for the Mebo 2.0.

Theoretical and untested Python code
This category includes code that can be built using existing Python libraries (like OpenCV and TensorFlow) to create more advanced, autonomous functionality. The core principle is combining the Mebo control library with external sensor data, primarily from the video feed.
1. Obstacle avoidance using computer vision
This code would process the Mebo's live camera feed using OpenCV to detect obstacles and command the robot to turn, as detailed in the previous response.

    Code components: python-nabot-mebo2 for robot control and OpenCV for image processing.
    Logic:
        Connect to the Mebo and its video stream.
        In a loop, capture video frames.
        Process the frames using a simple object detection or color filtering algorithm.
        If an object is detected in the path, send a command to the Mebo to turn.
        If the path is clear, send a command to move forward.

2. Object recognition and retrieval
This more advanced version of computer vision would train a machine learning model to recognize specific objects and then navigate to and grab them with the claw.

    Code components: python-nabot-mebo2, OpenCV, and a machine learning framework like TensorFlow or PyTorch.
    Logic:
        Train a model (e.g., a neural network) to recognize a specific object (e.g., a red ball).
        Connect to the Mebo and its video stream.
        In a loop, process the video feed with the trained model.
        Use the object's position within the frame to guide the Mebo's movement toward it.
        When the object is close enough, use the arm and claw commands to pick it up.

3. Voice command interface
This code would use a speech-to-text library to accept voice commands and translate them into Mebo actions.

    Code components: python-nabot-mebo2 for robot control and a speech recognition library like SpeechRecognition.
    Logic:
        Set up the program to listen for voice input from a microphone.
        Use the SpeechRecognition library to convert the audio into a text string.
        Use conditional logic to match keywords (e.g., "go forward," "stop," "grab object") to Mebo commands.
        Send the corresponding command to the robot.

4. Remote web interface
This code would use a Python web framework (e.g., Flask or Django) to create a web page that allows manual control and viewing the camera stream from any browser.

    Code components: python-nabot-mebo2 for control and Flask for the web server.
    Logic:
        Set up a Flask web server to host a simple web page.
        Create buttons or controls on the web page that, when clicked, send a request to the server.
        The server uses the python-nabot-mebo2 library to translate the web request into a Mebo command.
        Stream the Mebo's video feed to the web page.



Quote:
Found API endpoints and pages
The following endpoints and pages have been identified on the Mebo's web server at its IP address (typically 192.168.1.1):

    /status.html: Displays system status, including memory and baud rate.
    /system.html: Contains controls for Wi-Fi setup, resetting the video/controller board, and performing a factory reset.
    /upgrade.html: Shows firmware details and provides an interface for updating the firmware for both the video and controller boards.
    /nvram.html: A page with NVRAM parameters, though its full use has not been documented.

Identified API commands
The python-mebo library provides the most useful summary of the control commands by demonstrating how to use them with a Python script.
Movement commands

    m.move('n', speed=255, dur=1000): Moves the robot forward ('n') at a certain speed (speed) for a duration in milliseconds (dur).
    m.move('s', speed=255, dur=1000): Moves the robot backward ('s').
    m.move('e', speed=255, dur=1000): Rotates the robot right ('e').
    m.move('w', speed=255, dur=1000): Rotates the robot left ('w').
    m.arm.up(dur=1000): Moves the arm up for a duration.
    m.arm.down(dur=1000): Moves the arm down for a duration.
    m.arm.stop(): Stops arm movement.
    m.claw.open(dur=1000): Opens the claw for a duration.
    m.claw.close(dur=1000): Closes the claw for a duration.
    m.claw.stop(): Stops claw movement.

Camera controls

    Video Stream: The video stream is accessible via a URL. While the specific format is not provided, the python-mebo library connects to and uses the stream, indicating it can be accessed with the correct endpoint.

Other controls and hardware information

    Wi-Fi Connection: To control the Mebo, your device must be connected to its self-generated Wi-Fi hotspot, named "MEBO2 - xx-xx-xx".
    Reset Button: A physical reset button can be pressed to recalibrate motors or clear a forgotten Wi-Fi password. It is typically located on the underside of the robot.
    Hardware Modifications: The Mebo is a popular platform for modding. Hackers have replaced the internal controller board with a Raspberry Pi and a motor hat to gain full hardware control, bypassing the stock API entirely. This method provides the most granular control over individual motors.



Quote:
build app that autonomously controls mebo 2.0
Building an app for autonomous control of the Mebo 2.0 robot is possible because its internal API can be accessed and controlled with Python
, though the project requires a significant amount of coding and computer vision work. A company called Ximpatico already developed an AI add-on for the Mebo 2.0 called Nabot AI, and a Python API for the project is available on GitHub.
The project can be broken down into three main components:

    Controlling the hardware: Using a Python library to send commands to the robot via Wi-Fi.
    Sensing the environment: Using the robot's camera to perceive its surroundings.
    Implementing autonomy: Writing the logic that translates what the robot "sees" into actions.

Step 1: Use the Mebo Python API
You can use a reverse-engineered Python API to send commands to the Mebo 2.0.

    Install the library: The python-nabot-mebo2 library is an updated version of the original API and can be installed via pip.
    bash

    pip install python-nabot-mebo2

    Use code with caution.

Connect and test: Use the library to connect to the Mebo's Wi-Fi hotspot and send basic commands.
python

from mebo import Mebo
import time

# Initialize and connect to the Mebo
m = Mebo(auto_connect=True)

# Move forward for 2 seconds
m.move('n', speed=255, dur=2000)

# Wait for the move to finish
time.sleep(2)

# Open the arm claw
m.arm.claw.open(dur=1000)

Use code with caution.


Step 2: Implement computer vision for sensing
For the Mebo to act autonomously, it needs to understand its environment. Since the Mebo streams video, you can process that feed with a computer vision library like OpenCV to enable it to "see."

    Capture the video stream: Use Python to access the live video feed from the Mebo's camera.
    Process video with OpenCV:
        Set up a video capture object with OpenCV to receive the stream.
        Use OpenCV to detect obstacles or specific objects.
    Detect objects:
        For basic obstacle avoidance, you can use simple color filtering or edge detection.
        For more advanced scenarios, integrate a pre-trained machine learning model (e.g., from TensorFlow) to recognize objects.

Step 3: Write the autonomous control logic
The core of your autonomous app is the program that processes the camera input and uses the Mebo API to act on that information. The basic logic would be: "If I see an obstacle, turn. If not, go forward."
Here is an example of an obstacle avoidance loop:
python

import cv2
from mebo import Mebo
import time

# Initialize Mebo and video stream
m = Mebo(auto_connect=True)
cap = cv2.VideoCapture("rtsp://mebo_ip_address/stream") # Replace with Mebo's IP

try:
    while True:
        # Read a frame from the Mebo camera
        ret, frame = cap.read()
        if not ret:
            break
           
        # Add your OpenCV processing here
        # Example: Check if a red object is in the frame
        hsv = cv2.cvtColor(frame, cv2.COLOR_BGR2HSV)
        lower_red = (0, 50, 50)
        upper_red = (10, 255, 255)
        mask = cv2.inRange(hsv, lower_red, upper_red)
       
        # Calculate the number of red pixels
        num_red_pixels = cv2.countNonZero(mask)
       
        # Define the robot's logic
        if num_red_pixels > 5000:
            print("Obstacle detected! Turning left.")
            m.move('w', dur=1000) # Turn left
        else:
            print("Path clear. Moving forward.")
            m.move('n', speed=255, dur=1000) # Move forward
           
        time.sleep(1)
       
except KeyboardInterrupt:
    m.stop() # Ensure Mebo stops on exit
    cap.release()
    cv2.destroyAllWindows()

Use code with caution.
Potential project enhancements
Once you have the basics working, you can expand the project with more advanced features:

    Path planning: Use algorithms to chart the most efficient path to a target.
    Object recognition: Train a neural network to recognize and grab specific items with the robotic arm.
    Enhanced navigation: Implement features for improved mapping, localization, and navigation, similar to more advanced mobile robots.



https://github.com/Ximpatico/NabotPythonAPI

https://github.com/AtsushiSakai/PythonRobotics


--------------------
PAY NO ATTENTION TO THE MAN BEHIND THE CURTAIN get back to the garden

some came singing, some come to play, some come for keeping the dark away

Edited by ellomello (09/18/25 04:07 AM)

Extras: Filter Print Post Top
Re: AI controlled pet robots [Re: ellomello]
    #29347823 -

Quote:
A variety of apps and libraries are used to make simple GUIs, depending on the programming language and development goal. Here are some of the most prominent examples:
Python libraries

    Gooey: This Python library converts command-line Python programs into a graphical interface automatically, so you don't have to write GUI code. It's ideal for making scripts and developer tools more accessible to non-programmers.
    EasyGUI: A module for Python that provides a very simple, non-event-driven way to create GUI interactions through basic function calls. It's designed to be used without needing any knowledge of complex GUI frameworks like Tkinter.
    PySimpleGUI: A Python library built on top of other GUI frameworks like Tkinter, Qt, and Web. It greatly simplifies the process of creating user interfaces for Python applications, especially for beginners.
    Anvil: A Python GUI builder for web applications. It features a drag-and-drop designer and lets you build interactive web interfaces using only Python code, with no HTML or JavaScript required.
    NiceGUI: This Python library is for building web-based graphical interfaces. It is designed to be easy for beginners while also supporting advanced customizations for more complex projects.

General GUI builders and design tools

    Taipy: A low-code, drag-and-drop Python application builder for creating dynamic GUIs. It's built for Python developers who want a WYSIWYG (what-you-see-is-what-you-get) environment.
    Figma: A popular, web-based UI design and prototyping tool. It uses a visual, drag-and-drop interface that allows teams to collaborate on designs for websites, mobile apps, and other products.
    Visily: An AI-powered design software that uses drag-and-drop components and AI to help non-designers quickly create wireframes and prototypes from text prompts or diagrams.
    PyUiBuilder: A framework-agnostic, drag-and-drop GUI builder for Python that can generate code for multiple UI libraries, including Tkinter.

    EasyGUI — easygui 0.97 (2014-12-20) documentation
    EasyGUI. EasyGUI is a module for very simple, very easy GUI programming in Python. EasyGUI is different from other GUI generators in that EasyGUI is NOT event-d...
    favicon
    EasyGUI

A Python GUI Builder for the Web - Anvil Works
Build a Web GUI with Nothing but Python. Anvil is a GUI builder that lets you build web apps with nothing but Python. Drag and Drop Builder. Build your UI with ...
favicon
Anvil
An Introduction to Python Gooey - Fullstackdad - Medium
Nov 24, 2022 — Fullstackdad. Follow. 4 min read. Nov 24, 2022. 14. 1. Press enter or click to view image in full size. This is a demonstration of how to add a simple graphical...
favicon
Medium

Show all





--------------------
PAY NO ATTENTION TO THE MAN BEHIND THE CURTAIN get back to the garden

some came singing, some come to play, some come for keeping the dark away

Extras: Filter Print Post Top
Jump to top Pages: 1

Shop: MagicBag.co All-In-One Bags That Don't Suck, Certified Organic All-In-One Grow Bags   Myyco.com Golden Teacher Liquid Culture For Sale   North Spore North Spore Mushroom Grow Kits & Cultivation Supplies


Extra information
You cannot start new topics / You cannot reply to topics
HTML is disabled / BBCode is enabled
Moderator: Ythan, Anno, Thor, Link, Seuss
185 topic views. 0 members, 1 guests and 0 web crawlers are browsing this forum.
[ Show Images Only | Sort by Score | Print Topic ]
Search this thread:

Copyright 1997-2026 Mind Media. Some rights reserved.

Generated in 0.022 seconds spending 0.004 seconds on 14 queries.