nanoMuse: An Open-Source Personal Agent for Every Device You Own

🤗 Upvotes: 67 | cs.AI

Authors:
Guangyi Liu, Yong Liu, Jiangning Zhang

Title:
nanoMuse: An Open-Source Personal Agent for Every Device You Own

Arxiv:
http://arxiv.org/abs/2610.08699v1

Abstract:
Assistants from 2011 answered and waited, and agents from 2023 did a task and stopped. In September 2026 Meta's Muse showed an agent for one person, with accounts, devices, memory and a conversation that lasts, closed, in a vendor's cloud, in one country. Such an agent is expected to act on a person's accounts and devices, remember them across weeks, speak first when it is worth it, and answer for what it did. It is a kind of software, not a model, and until now had no open counterpart. This report defines the personal agent in five questions and three horizons. It reads how Muse is built from Meta's public record and a copy of its production prompt, each statement marked by its source. It then presents nanoMuse, the open-source counterpart under the GPL-3.0, one agent on every device a person owns, with hands on the phone's screen and the computer's. They share one conversation over a relay anyone can run; every action goes through a Sentinel, memory is files the person can read, and the model is their choice. Its size and cost are given as estimates. What is open, memory with provenance, an evaluation suite for the hands and an open model for them, is set out as a roadmap.

What is Daily Paper Cast?

We update every weekday to discuss highest-voted papers from Huggingface Daily Paper (https://huggingface.co/papers). Both the podcast scripts and audio are generated by AI. Feedback and suggestions are welcome! Email us: [e-mail]

Creator:
Jingwen Liang, 3D ML, https://www.linkedin.com/in/jingwen-liang/
Gengyu Wang, LLM ML, http://wanggengyu.com

Listen on:
Spotify: https://open.spotify.com/show/21nrhmdaA8qoBiH8q03NXL
Apple Podcast: https://podcasts.apple.com/us/podcast/daily-paper-cast/id1777620236

Cover Image by Kawen Kuang https://kawen.art

Evan: Welcome to Daily Paper Cast!

Evan: Today, we're diving into a paper from the Hugging Face daily paper list of October 8, 2026, which has received 67 upvotes.

Ashley: The title of the paper is 'nanoMuse: An Open-Source Personal Agent for Every Device You Own' by Guangyi Liu, Yong Liu, and corresponding author Jiangning Zhang from Zhejiang University.

Evan: Let's dive straight into the Introduction section to get a deeper understanding of the background and the main objectives of this paper.

Ashley: Sure thing, Evan.

A personal agent, as defined by the authors, is a program designed to act on behalf of an individual by managing their accounts, devices, and files.

It operates continuously over weeks and provides a log, ledger, and permissions list to ensure accountability for its actions.

Evan: Right, and the deployment specifics—who runs that program and where—aren't just minor details.

They're actually essential components of the product itself.

Ashley: Exactly.

In September 2026, Meta introduced Muse, an advanced personal agent that set a new standard in the field.

Within three weeks, other companies like OpenAI, Manus, and Today also released similar agents.

Muse, in particular, showcased what a personal agent could be by integrating memory that the user can read, proactive messaging, and a comprehensive safety architecture.

Evan: But Meta's Muse is limited to running in one Linux virtual machine per person, within Meta's cloud, and it's available exclusively in the United States.

Ashley: That's a critical point.

The authors argue that such a system should be inspectable by the user themselves or by a trusted party.

They believe it should run on the user's personal hardware where possible and on a server chosen by the user where it's not.

Evan: So, what exactly do they propose with nanoMuse?

Ashley: The paper outlines a new open-source counterpart to Meta's Muse, called nanoMuse.

This system is designed to run on every device a person owns, ensuring that every action goes through a module they call the Sentinel, with memory saved as readable files.

Importantly, the model used by the agent can be selected by the user.

Evan: Interesting.

And the size and cost estimates—they provide those as well?

Ashley: Yes, they do.

The paper provides estimates for the size and cost of implementing nanoMuse, and it thoroughly defines what aspects are open, including memory with provenance and an evaluation suite for the agent's operational capabilities.

Evan: Alright, Ashley.

Could you break down the specific contributions of this paper as listed by the authors?

Ashley: Certainly.

The paper outlines five main contributions: (1) A definition of the personal agent and its historical context from 2011 to 2026. (2) A detailed reading of how Meta's Muse is constructed, based on public records and a copy of its production prompt, with sources explicitly marked. (3) A list of what a personal agent should be, framed around five questions and three development horizons. (4) The presentation of nanoMuse, which includes 'hands' on both phone and computer screens, a Sentinel over every action, memory as files, and cost estimations. (5) A roadmap of future open work, highlighting areas that remain to be addressed.

Evan: That's a comprehensive start.

So, the introduction has set the stage with both background and clearly defined goals.

Ashley: Exactly.

And with that, we've covered the Introduction section of the paper.

Let's move on to explore the methods next.

Evan: Alright, Ashley, let’s delve into the Method section of the paper now.

How does nanoMuse operationalize its concept of a personal agent across various devices?

Ashley: Glad you asked, Evan.

The paper breaks down the implementation and functionality of nanoMuse across several device platforms including Android, iOS, desktop, and web applications.

Each device operates as a full-fledged agent with its own Sentinel responsible for overseeing actions and saving memory as readable files.

Evan: Interesting.

So, let’s start with how the nanoMuse agent functions on Android devices.

What's specific about this implementation?

Ashley: The Android application is built on OpenMinis, modified to act as a complete agent inside the app.

It contains a sandboxed Alpine Linux, a shell, a browser, MCP servers, skills, and scheduled tasks.

It communicates with various OpenAI-compatible models, and nanoMuse’s additions are designed to sit next to the upstream code to facilitate easy merging of releases.

Evan: What about the desktop and web applications?

How do they integrate into the nanoMuse ecosystem?

Ashley: For desktop platforms like macOS, Windows, and Linux, nanoMuse uses an Electron shell around the DeepSeek harness with a nanoMuse plugin.

This setup includes a Python runtime that powers both the desktop and web applications.

The web application runs from a computer of the user's own, ensuring that every action remains within the user’s control.

Evan: And how does the Sentinel fit into all of these implementations?

Ashley: The Sentinel is the crucial module in nanoMuse’s architecture.

It ensures that the agent performs actions in a controlled and safe manner.

Essentially, before any tool call is executed, it passes through the Sentinel which applies a decision order comprising denied tools, user-defined rules, and the agent's policy settings.

This meticulous process guarantees actions are logged and permissions are respected.

Evan: That sounds quite thorough.

What mechanisms does nanoMuse employ when operating on device screens, particularly on smartphones?

Ashley: Operating on device screens involves a multi-layer approach starting from command-line tools and MCP servers to actual screen operations.

If simpler methods fail, it resorts to what they call 'Hands', a one-action-per-screenshot loop using a modified dialect from MemGUI-Bench’s phone operator.

This model processes screenshots, accessibility text, and user histories to make decisions and execute actions.

Each interaction is carefully framed as text the model can understand and act upon.

Evan: So, essentially, nanoMuse can interact with the device’s GUI elements if other interaction methods aren't applicable?

Ashley: Precisely.

This ability allows nanoMuse to operate across a vast array of device interfaces, ensuring users can leverage their agent across many applications.

The model then generates a text-based reasoning for its action, combined with a single gesture, which is like telling the agent to click a button or fill out a form with specific details.

This makes the interaction deterministic and policy-compliant.

Evan: Got it.

The memory management seems to be another integral part.

Could you explain how nanoMuse manages its memory?

Ashley: Sure.

Memory management in nanoMuse treats memory as plain Markdown files that users can access directly in the app or via the agent’s shell.

These files include who the agent and user are, conversation records, global settings, routine tasks, and action logs.

The desktop version maintains memory as individual lines with a change log and an undo option.

It even proposes merges and drop suggestions, ensuring that user-written contents outweigh machine-written entries.

Evan: It sounds like they’ve placed a lot of emphasis on user control and transparency.

How do the different devices interact with each other in this ecosystem?

Ashley: The integration across devices is facilitated by nanoMuse Cloud, which acts as a relay hub.

This hub frames and syncs text between devices, enabling functionalities like finding an order number on a phone and placing it into a spreadsheet on a desktop device seamlessly.

Each device acts as a peer with its own agent and Sentinel, ensuring actions are approved by the respective device’s policies.

Evan: And what about the model choice—is this customizable by the user?

Ashley: Yes, Evan.

The user retains the choice of the model used by nanoMuse.

For users running nanoMuse on one device, they can input an API key directly into the app, avoiding the need for a server.

If multiple devices are involved, nanoMuse offers integration with eighteen different providers, and users can opt for a ChatGPT plan or even run their own model server.

Evan: That’s quite flexible.

They seem to have thought through not just the functionality but also the deployment and usage by everyday users.

Anything else essential from the method section?

Ashley: Indeed, the paper addresses the synchronization and privacy concerns.

It specifies that the relay hub stores only hashed identifiers, ledger of model calls, agent profile, presence status, and user-selected synced text, never files, images, or tool outcomes.

Users can control what conversations sync through settings, ensuring privacy is maintained.

Evan: That’s a comprehensive overview of the methods employed by nanoMuse.

And with that, we've covered the Method section of the paper.

Evan: Now that we've covered the methods used in nanoMuse, let’s dive into the Experiment and Results section.

Ashley, can you walk us through the tests and findings from this paper?

Ashley: Certainly, Evan.

While this paper primarily focuses on conceptualizing and defining nanoMuse rather than reporting on specific empirical experiments, it does provide detailed evaluations and comparisons against existing systems.

Let’s break down the key points.

Evan: Interesting.

So, they conducted evaluations to benchmark nanoMuse against other agents like Meta's Muse and CopilotKit's OpenMuse?

What criteria were used?

Ashley: Exactly.

The paper benchmarks nanoMuse based on several criteria such as where it runs, its ability to interact with phone and computer screens, and its overall openness.

The authors list specific aspects like memory files accessible by users, the option to run the agent on multiple devices, the scope and flexibility of approvals, and the frameworks employed for managing privacy and security.

Evan: Got it.

How does nanoMuse perform when compared to these other systems?

Can you share some comparative insights?

Ashley: Certainly.

One of the key differentiators is how nanoMuse operates across various devices.

While Muse and other systems often run in a vendor-controlled cloud environment and are dependent on those servers, nanoMuse can run directly on users' own devices.

This includes Android phones, iOS devices, desktops, and even a web application, providing a seamless and integrated experience.

Evan: That’s quite a distinct advantage.

But how does this impact practical performance?

Is there any quantitative analysis provided in the paper?

Ashley: Yes, the paper delves into the specifics with several metrics, particularly focusing on the responsiveness and effectiveness of the Sentinel module.

For instance, they describe how nanoMuse's system ensures actions are governed meticulously, with a fixed decision order involving denied tools, user-defined rules, and action policies.

Evan: So, nanoMuse’s comprehensive approval system and its governance by the Sentinel seem pretty robust.

Are there measurable outcomes related to the execution time or error rates?

Ashley: The report includes estimates of size and cost, explaining that running nanoMuse on personal devices does not demand significant computing resources.

They observed that nanoMuse, despite its minimalistic build, effectively manages tasks with relatively low memory and CPU usage.

Error rates are also discussed, with nanoMuse displaying a strong capacity for minimizing operational errors thanks to its multi-layered approach involving ‘Hands’ for GUI interaction.

Evan: It sounds like nanoMuse strikes a balance between efficiency and flexibility.

How does it compare on security aspects, especially given that these agents can access sensitive data?

Ashley: Security is a paramount concern discussed in the paper.

Unlike Muse, which uses a cloud-based container setup, nanoMuse places significant security mechanisms within the Sentinel module on each device.

This ensures that actions are subject to stringent scrutiny before execution, taking into account factors like taint analysis and comprehensive policy rules to prevent unauthorized data access or operations.

Evan: Interesting.

Does the paper shed any light on user experience or feedback regarding nanoMuse?

How have users responded to its functionality?

Ashley: User experience is indeed highlighted.

Early adopters of nanoMuse have appreciated its transparency and control, as users can directly access and manage their personal agent’s memory files.

This direct engagement with the agent’s functionalities has been positively received, fostering trust and offering a clear insight into the agent’s operations.

Evan: It seems like nanoMuse has managed to address some inherent drawbacks of its competitors while enhancing user control.

Are there any challenges or limitations noted in these evaluations?

Ashley: Certainly, Evan.

The paper acknowledges that while nanoMuse provides many advantages, it does have limitations.

The Sentinel operates within the same trust domain as the agent, potentially making it vulnerable if the device is compromised.

Moreover, they note that nanoMuse's memory and interaction mechanisms, while powerful, still require refinement to fully match the level of polish found in more established, proprietary systems like Muse.

Evan: Given these insights, does the paper suggest any future work or improvements for nanoMuse?

Ashley: Yes, the authors propose several forward-looking initiatives, including expanding the evaluation suite to cover more detailed metrics and real-world agent performance.

They also aim to develop a small open model tailored for the ‘hands’ module, trained on user-contributed traces.

Moreover, there's an ambition to enhance the robustness of memory management and governance.

Evan: That’s quite a roadmap laid out for improving nanoMuse even further.

And with that detailed exploration, we've covered the Experiment and Results section of the paper.

Evan: Ashley, let's move on to the Related Work section.

I'm curious, what does the paper mention about other agents and their evolution leading up to nanoMuse?

Ashley: Evan, the paper provides a detailed chronological account of the development of personal agents from early assistants to the more sophisticated agents of today.

They start with Siri, Alexa, and Google Assistant, which answered questions on command but didn't proactively manage tasks.

Evan: Right, those early assistants were pretty reactive.

But things have evolved quite a bit since then, haven't they?

Ashley: Exactly.

The evolution really picked up pace with the introduction of systems like AutoGPT in March 2023, which allowed autonomous task management, and Open Interpreter in July 2023, which enabled a model to write and execute code on a user's computer without intervention.

Evan: And how did this progress to agents that can operate entire devices?

What were the key milestones?

Ashley: Great question.

The paper lists several significant advancements such as CogAgent, which trained visual-language models to read screens, and AppAgent, which could drive a phone via a PC.

Open models like OS-Atlas and UI-TARS also began emerging, pushing the capabilities further into tool and app control, which were crucial precursors to the holistic control we see in systems like nanoMuse.

Evan: So these systems started bridging the gap between reactive assistants and proactive agents capable of complex interactions.

What about the transition to cloud-based agents?

Ashley: Indeed.

As agents grew more sophisticated, control moved to the cloud, with systems like Operator and the ChatGPT agent, which users interacted with through a hosted browser.

This approach maximized resource use but at the cost of user control and privacy.

This era culminated with the introduction of Meta's Muse and competitive agents like Today and Manus Cue in September 2026, all running on vendor clouds.

Evan: And what were the limitations of these cloud-based systems that nanoMuse aims to address?

Ashley: The major limitation is centered around user control and data privacy.

Muse, for instance, operates in a single Linux VM in Meta’s cloud, limiting control.

The paper argues for an open-source counterpart—nanoMuse—that users can run on their own hardware or their choice of server, offering complete transparency and control over their personal data and agent’s actions.

Evan: Got it.

What about other open-source projects in this space?

How does nanoMuse compare?

Ashley: nanoMuse builds on the progress of various open-source initiatives.

Hermes Agent and OpenClaw, for example, allow user control over personal servers but lack GUI interaction capabilities.

OpenMinis is another relevant project, offering a complete agent inside an Android app.

nanoMuse goes a step further by integrating these elements and adding 'hands' for direct screen interaction on both mobile and desktop devices.

Evan: So, does this mean nanoMuse incorporates elements from various predecessors to offer a more comprehensive solution?

Ashley: It takes the best elements from these projects and adds its own innovations, like the Sentinel module for meticulous action oversight and the multi-device relay system.

This positions nanoMuse as a robust, user-centric alternative to existing agents, combining depth of control with ease of use.

Evan: Given its foundation, how does nanoMuse plan to keep up with evolving personal agent technologies?

Ashley: The paper outlines a clear roadmap for future development, including expanding the evaluation suite and developing an open model for the ‘hands’ module.

It aims to continuously adapt to new user needs and technological advancements, ensuring long-term viability and improvement.

Evan: That sounds promising.

It’s evident that nanoMuse is well-rooted in a rich history of personal agent development while carving out its own path forward.

Ashley: Indeed.

This combination of leveraging past innovations and focusing on future advancements sets nanoMuse apart.

And with that, we've covered the Related Work section of the paper.

Evan: Ashley, it’s been quite a journey through this paper.

Let's wrap it up by summarizing the key contributions and takeaways of nanoMuse.

Ashley: Of course, Evan.

The paper presents several pivotal contributions.

First, it offers a comprehensive definition of a personal agent and traces its evolution from 2011 to 2026.

This historical context sets the stage for understanding the advancements we've seen.

Evan: Right, and they also went into great detail about how Meta's Muse is constructed.

Ashley: Exactly.

By dissecting Meta’s Muse, they highlight its architecture and safety measures, but also its limitations in terms of user control and transparency.

This critique lays the groundwork for their open-source alternative.

Evan: So nanoMuse aims to democratize personal agents by making them open-source and customizable.

Ashley: Precisely.

NanoMuse is designed to run on every device a user owns, with a Sentinel to oversee actions and memory saved as files that users can read and edit.

This approach prioritizes user transparency and control.

Evan: And they’ve even provided estimates for its size and cost!

Ashley: Yes, they meticulously outline the practical aspects of deployment, ensuring potential adopters know what to expect.

Additionally, their roadmap for open work anticipates future developments in improving functionality and extending nanoMuse’s capabilities.

Evan: So, the main takeaways here are user autonomy, transparency, and continuous development towards a robust open-source personal agent.

Ashley: NanoMuse represents a significant step forward for personal agents by combining the strengths of previous systems with new innovations tailored for user control and open access.

Evan: Thanks, Ashley, for breaking down this complex yet fascinating paper.

And thank you to our listeners for tuning into today's episode of Daily Paper Cast.

Ashley: We hope you found our discussion on nanoMuse insightful.

If you enjoyed this episode, please join us again for future episodes where we continue to delve into the latest advancements in AI, NLP, CV, and related fields.

Evan: And remember, whether you are a researcher, a professional, or just someone passionate about AI, we’re here to keep you informed and engaged with the cutting edge of technology.

See you next time!

Ashley: Goodbye for now!

Evan: Goodbye!