mirror of
https://github.com/tinyhumansai/openhuman.git
synced 2026-07-27 21:08:00 +00:00
feat(local_ai): default to cloud fallback on <8GB RAM devices (#589)
* Enhance draft message handling in streaming edits
- Introduced a new `draft_sent` field in the `StreamingState` struct to track when a draft message has been posted, decoupling the existence of a draft from the ability to edit it.
- Updated the `flush_streaming_edit` function to set `draft_sent` upon posting a draft, ensuring that duplicate messages are not sent if the backend fails to return an ID.
- Modified the `finalize_channel_reply` function to prevent sending a fresh message if a draft was already posted without an ID, improving user experience by avoiding duplicate bubbles.
These changes enhance the robustness of message handling during streaming edits, ensuring a smoother interaction for users.
* feat(onboarding): enhance LocalAIStep for low-RAM device handling
- Added logic to determine if the device is below the RAM threshold, defaulting to a cloud AI fallback if necessary.
- Introduced a new UI for low-RAM devices, informing users about the cloud mode and providing options to continue with cloud AI or force-enable local AI.
- Updated the `ensureRecommendedLocalAiPresetIfNeeded` function to return a `recommend_disabled` flag based on device RAM.
- Enhanced tests to cover new cloud fallback UI and local AI consent handling.
These changes improve the onboarding experience by providing clearer options based on device capabilities.
* style: apply prettier formatting to LocalAIStep
* feat(local_ai): unlock model selection + add Disabled/cloud fallback option
- Remove MVP ceiling that clamped selection to the 2-4 GB tier, in bootstrap
and in the apply_preset RPC — users can now pick any tier.
- Add special "disabled" tier string to apply_preset that toggles
config.local_ai.enabled = false so the app uses the cloud summarizer.
- Presets RPC now returns local_ai_enabled so the UI can render the
currently-active state correctly.
- Rewrite DeviceCapabilitySection: tiers are now clickable buttons that
call apply_preset; adds a "Disabled — Cloud fallback" card at the top
marked "Recommended" on low-RAM devices and "Active" when enabled=false.
- Remove the MVP message copy from the model tier panel.
- Remove unread-count badge from the bottom tab bar.
* refactor(onboarding): streamline LocalAIStep and update local AI preset handling
- Removed the `ensureRecommendedLocalAiPresetIfNeeded` function call from `LocalAIStep`, replacing it with `openhumanLocalAiPresets` to directly fetch preset information.
- Updated the logic in `ensureRecommendedLocalAiPresetIfNeeded` to ensure the recommended tier is applied correctly based on user consent, improving clarity in the local AI setup process.
- Enhanced tests to mock the new `openhumanLocalAiPresets` function, ensuring coverage for low-RAM device scenarios and local AI consent handling.
- Updated documentation in the `schemas.rs` file to reflect changes in the data structure returned by the presets RPC.
These changes improve the onboarding experience by providing a more direct and efficient method for managing local AI presets based on device capabilities.
* test(LocalAIStep): improve mock implementation for local AI presets
- Refactored the mock for `openhumanLocalAiPresets` to enhance readability and maintainability.
- Ensured the mock returns consistent device information and preset details for testing scenarios.
- This change supports better test coverage and clarity in the LocalAIStep component's behavior during onboarding.
* fix(local_ai): preserve hadSelectedTier semantics in bootstrap return
hadSelectedTier / selectedTier represent the incoming state ("was a tier
already selected before this call"), not the post-apply state. Setting
both to the just-applied tier broke the existing localAiBootstrap
contract and the unit test that encodes it.
The Rust-side persistence fix is independent: removing the
recommend_disabled short-circuit ensures openhumanLocalAiApplyPreset
actually writes the selected tier to disk on the opt-in path, so
config_with_recommended_tier_if_unselected() honors the user's choice.
This commit is contained in:
Steven Enamakel
@@ -37,7 +37,7 @@ const LocalModelPanel = () => {
|
||||
const [presetsData, setPresetsData] = useState<PresetsResponse | null>(null);
|
||||
const [presetsLoading, setPresetsLoading] = useState(true);
|
||||
const [presetError, setPresetError] = useState('');
|
||||
const [presetSuccess] = useState<ApplyPresetResult | null>(null);
|
||||
const [presetSuccess, setPresetSuccess] = useState<ApplyPresetResult | null>(null);
|
||||
|
||||
const progress = useMemo(() => {
|
||||
const downloadProgress = progressFromDownloads(downloads);
|
||||
@@ -134,6 +134,11 @@ const LocalModelPanel = () => {
|
||||
presetError={presetError}
|
||||
presetSuccess={presetSuccess}
|
||||
formatRamGb={formatRamGb}
|
||||
onPresetApplied={result => {
|
||||
setPresetSuccess(result);
|
||||
void loadPresets();
|
||||
void loadStatus();
|
||||
}}
|
||||
/>
|
||||
|
||||
{/* Simplified download status */}
|
||||
|
||||
@@ -1,4 +1,10 @@
|
||||
import type { ApplyPresetResult, PresetsResponse } from '../../../../utils/tauriCommands';
|
||||
import { useState } from 'react';
|
||||
|
||||
import {
|
||||
type ApplyPresetResult,
|
||||
openhumanLocalAiApplyPreset,
|
||||
type PresetsResponse,
|
||||
} from '../../../../utils/tauriCommands';
|
||||
|
||||
interface DeviceCapabilitySectionProps {
|
||||
presetsData: PresetsResponse | null;
|
||||
@@ -6,15 +12,40 @@ interface DeviceCapabilitySectionProps {
|
||||
presetError: string;
|
||||
presetSuccess: ApplyPresetResult | null;
|
||||
formatRamGb: (bytes: number) => string;
|
||||
onPresetApplied?: (result: ApplyPresetResult) => void;
|
||||
}
|
||||
|
||||
const DISABLED_TIER_ID = 'disabled';
|
||||
|
||||
const DeviceCapabilitySection = ({
|
||||
presetsData,
|
||||
presetsLoading,
|
||||
presetError,
|
||||
presetSuccess,
|
||||
formatRamGb,
|
||||
onPresetApplied,
|
||||
}: DeviceCapabilitySectionProps) => {
|
||||
const [applying, setApplying] = useState<string | null>(null);
|
||||
const [applyError, setApplyError] = useState<string>('');
|
||||
const [applySuccess, setApplySuccess] = useState<ApplyPresetResult | null>(null);
|
||||
|
||||
const isDisabledActive = presetsData ? presetsData.local_ai_enabled === false : false;
|
||||
|
||||
const handleApply = async (tierId: string) => {
|
||||
setApplying(tierId);
|
||||
setApplyError('');
|
||||
try {
|
||||
const result = await openhumanLocalAiApplyPreset(tierId);
|
||||
setApplySuccess(result);
|
||||
onPresetApplied?.(result);
|
||||
} catch (err) {
|
||||
const msg = err instanceof Error ? err.message : 'Failed to apply preset';
|
||||
setApplyError(msg);
|
||||
} finally {
|
||||
setApplying(null);
|
||||
}
|
||||
};
|
||||
|
||||
return (
|
||||
<section className="space-y-3">
|
||||
<h3 className="text-sm font-semibold text-stone-900">Model Tier</h3>
|
||||
@@ -63,21 +94,51 @@ const DeviceCapabilitySection = ({
|
||||
|
||||
{presetsData && (
|
||||
<div className="space-y-2">
|
||||
<div className="rounded-lg border border-primary-200 bg-primary-50 p-3 text-xs text-primary-700">
|
||||
The local AI model is fixed for the MVP release. Broader model options will be available
|
||||
in a future update.
|
||||
</div>
|
||||
{/* Disabled — Cloud fallback card (always available, recommended on low-RAM) */}
|
||||
<button
|
||||
type="button"
|
||||
onClick={() => void handleApply(DISABLED_TIER_ID)}
|
||||
disabled={applying !== null}
|
||||
className={`w-full text-left rounded-lg border p-3 transition-colors ${
|
||||
isDisabledActive
|
||||
? 'border-primary-400 bg-primary-50'
|
||||
: 'border-stone-200 bg-stone-50 hover:bg-stone-100'
|
||||
} ${applying !== null ? 'opacity-60 cursor-not-allowed' : 'cursor-pointer'}`}>
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="flex items-center gap-2">
|
||||
<span className="text-sm font-semibold text-stone-900">Disabled</span>
|
||||
{isDisabledActive && (
|
||||
<span className="px-1.5 py-0.5 text-[10px] font-medium rounded bg-primary-50 text-primary-600 uppercase tracking-wide">
|
||||
Active
|
||||
</span>
|
||||
)}
|
||||
{presetsData.recommend_disabled && !isDisabledActive && (
|
||||
<span className="px-1.5 py-0.5 text-[10px] font-medium rounded bg-amber-50 text-amber-700 uppercase tracking-wide">
|
||||
Recommended
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
<span className="text-xs text-stone-500">0 GB</span>
|
||||
</div>
|
||||
<div className="text-xs text-stone-500 mt-1">
|
||||
Fallback to the cloud summarizer model. No local download or Ollama install required.
|
||||
</div>
|
||||
</button>
|
||||
|
||||
{presetsData.presets.map(preset => {
|
||||
const isCurrent = preset.tier === presetsData.current_tier;
|
||||
const isLocked = !isCurrent;
|
||||
const isCurrent = !isDisabledActive && preset.tier === presetsData.current_tier;
|
||||
const isApplying = applying === preset.tier;
|
||||
return (
|
||||
<div
|
||||
<button
|
||||
type="button"
|
||||
key={preset.tier}
|
||||
className={`w-full text-left rounded-lg border p-3 ${
|
||||
onClick={() => void handleApply(preset.tier)}
|
||||
disabled={applying !== null}
|
||||
className={`w-full text-left rounded-lg border p-3 transition-colors ${
|
||||
isCurrent
|
||||
? 'border-primary-400 bg-primary-50'
|
||||
: 'border-stone-200 bg-stone-50 opacity-50'
|
||||
}`}>
|
||||
: 'border-stone-200 bg-stone-50 hover:bg-stone-100'
|
||||
} ${applying !== null && !isApplying ? 'opacity-60 cursor-not-allowed' : 'cursor-pointer'}`}>
|
||||
<div className="flex items-center justify-between">
|
||||
<div className="flex items-center gap-2">
|
||||
<span className="text-sm font-semibold text-stone-900">{preset.label}</span>
|
||||
@@ -86,9 +147,9 @@ const DeviceCapabilitySection = ({
|
||||
Active
|
||||
</span>
|
||||
)}
|
||||
{isLocked && (
|
||||
<span className="px-1.5 py-0.5 text-[10px] font-medium rounded bg-stone-100 text-stone-400 uppercase tracking-wide">
|
||||
Coming soon
|
||||
{isApplying && (
|
||||
<span className="px-1.5 py-0.5 text-[10px] font-medium rounded bg-stone-100 text-stone-500 uppercase tracking-wide">
|
||||
Applying…
|
||||
</span>
|
||||
)}
|
||||
</div>
|
||||
@@ -104,11 +165,11 @@ const DeviceCapabilitySection = ({
|
||||
: preset.vision_model_id || preset.vision_mode}{' '}
|
||||
· Target RAM: {preset.target_ram_gb} GB
|
||||
</div>
|
||||
</div>
|
||||
</button>
|
||||
);
|
||||
})}
|
||||
|
||||
{presetsData.current_tier === 'custom' && (
|
||||
{presetsData.current_tier === 'custom' && !isDisabledActive && (
|
||||
<div className="rounded-lg border border-amber-200 bg-amber-50 p-3 text-xs text-amber-700">
|
||||
You are using custom model IDs that do not match any built-in preset.
|
||||
</div>
|
||||
@@ -116,12 +177,19 @@ const DeviceCapabilitySection = ({
|
||||
</div>
|
||||
)}
|
||||
|
||||
{applyError && <div className="text-xs text-red-600">{applyError}</div>}
|
||||
{presetError && !(!presetsLoading && !presetsData) && (
|
||||
<div className="text-xs text-red-600">{presetError}</div>
|
||||
)}
|
||||
{presetSuccess && (
|
||||
{(applySuccess ?? presetSuccess) && (
|
||||
<div className="text-xs text-green-700">
|
||||
Applied {presetSuccess.applied_tier} tier: {presetSuccess.chat_model_id}
|
||||
{(applySuccess ?? presetSuccess)?.applied_tier === DISABLED_TIER_ID
|
||||
? 'Local AI disabled — using cloud fallback.'
|
||||
: `Applied ${(applySuccess ?? presetSuccess)?.applied_tier} tier${
|
||||
(applySuccess ?? presetSuccess)?.chat_model_id
|
||||
? `: ${(applySuccess ?? presetSuccess)?.chat_model_id}`
|
||||
: ''
|
||||
}`}
|
||||
</div>
|
||||
)}
|
||||
</section>
|
||||
|
||||
@@ -1,6 +1,7 @@
|
||||
import { useCallback, useRef } from 'react';
|
||||
import { useCallback, useEffect, useRef, useState } from 'react';
|
||||
|
||||
import { bootstrapLocalAiWithRecommendedPreset } from '../../../utils/localAiBootstrap';
|
||||
import { openhumanLocalAiPresets } from '../../../utils/tauriCommands';
|
||||
import OnboardingNextButton from '../components/OnboardingNextButton';
|
||||
|
||||
/* ---------- component ---------- */
|
||||
@@ -13,6 +14,25 @@ interface LocalAIStepProps {
|
||||
|
||||
const LocalAIStep = ({ onNext, onBack: _onBack, onDownloadError }: LocalAIStepProps) => {
|
||||
const downloadStartedRef = useRef(false);
|
||||
const [recommendDisabled, setRecommendDisabled] = useState<boolean | null>(null);
|
||||
|
||||
useEffect(() => {
|
||||
let cancelled = false;
|
||||
// Read-only probe: never apply/persist a preset from the mount effect.
|
||||
// Preset application lives in handleConsent via bootstrapLocalAiWithRecommendedPreset.
|
||||
openhumanLocalAiPresets()
|
||||
.then(presets => {
|
||||
if (!cancelled) {
|
||||
setRecommendDisabled(presets.recommend_disabled ?? false);
|
||||
}
|
||||
})
|
||||
.catch(() => {
|
||||
if (!cancelled) setRecommendDisabled(false);
|
||||
});
|
||||
return () => {
|
||||
cancelled = true;
|
||||
};
|
||||
}, []);
|
||||
|
||||
const handleConsent = useCallback(() => {
|
||||
if (downloadStartedRef.current) return;
|
||||
@@ -29,6 +49,83 @@ const LocalAIStep = ({ onNext, onBack: _onBack, onDownloadError }: LocalAIStepPr
|
||||
onNext({ consentGiven: true, downloadStarted: true });
|
||||
}, [onNext, onDownloadError]);
|
||||
|
||||
const handleSkip = useCallback(() => {
|
||||
console.debug('[LocalAIStep] skipping local AI — using cloud fallback');
|
||||
onNext({ consentGiven: false, downloadStarted: false });
|
||||
}, [onNext]);
|
||||
|
||||
// Still probing device — show nothing yet.
|
||||
if (recommendDisabled === null) {
|
||||
return null;
|
||||
}
|
||||
|
||||
// Low-RAM device: show cloud fallback option as the primary path.
|
||||
if (recommendDisabled) {
|
||||
return (
|
||||
<div className="rounded-2xl border border-stone-200 bg-white p-8 shadow-soft animate-fade-up">
|
||||
<div className="flex flex-col items-center mb-5">
|
||||
<div className="flex h-16 w-16 items-center justify-center rounded-full bg-primary-50 mb-3">
|
||||
<svg
|
||||
className="h-8 w-8 text-primary-500"
|
||||
fill="none"
|
||||
viewBox="0 0 24 24"
|
||||
stroke="currentColor"
|
||||
strokeWidth={1.5}>
|
||||
<path
|
||||
strokeLinecap="round"
|
||||
strokeLinejoin="round"
|
||||
d="M2.25 15a4.5 4.5 0 004.5 4.5H18a3.75 3.75 0 001.332-7.257 3 3 0 00-3.758-3.848 5.25 5.25 0 00-10.233 2.33A4.502 4.502 0 002.25 15z"
|
||||
/>
|
||||
</svg>
|
||||
</div>
|
||||
<h1 className="text-xl font-bold mb-2 text-stone-900">AI — Cloud Mode</h1>
|
||||
<p className="text-stone-600 text-sm text-center">
|
||||
Your device has limited RAM, so we'll use a fast, lightweight cloud model for AI
|
||||
features. You can switch to local AI later in Settings.
|
||||
</p>
|
||||
</div>
|
||||
|
||||
<div className="space-y-2 mb-5">
|
||||
<div className="rounded-xl border border-primary-200 bg-primary-50 px-3 py-2">
|
||||
<p className="text-xs text-stone-700">
|
||||
<span className="font-semibold">Fast & lightweight</span>
|
||||
<span className="text-stone-600">
|
||||
— uses a cheap cloud summarizer model with minimal latency.
|
||||
</span>
|
||||
</p>
|
||||
</div>
|
||||
<div className="rounded-xl border border-stone-200 bg-stone-50 px-3 py-2">
|
||||
<p className="text-xs text-stone-700">
|
||||
<span className="font-semibold">No downloads needed</span>
|
||||
<span className="text-stone-600">
|
||||
— no large model files or Ollama install required.
|
||||
</span>
|
||||
</p>
|
||||
</div>
|
||||
<div className="rounded-xl border border-amber-200 bg-amber-50 px-3 py-2">
|
||||
<p className="text-xs text-stone-700">
|
||||
<span className="font-semibold">Requires internet</span>
|
||||
<span className="text-stone-600">
|
||||
— AI features need an active connection. You can opt into local AI in Settings
|
||||
if preferred.
|
||||
</span>
|
||||
</p>
|
||||
</div>
|
||||
</div>
|
||||
|
||||
<OnboardingNextButton label="Continue with Cloud" onClick={handleSkip} />
|
||||
|
||||
<button
|
||||
type="button"
|
||||
onClick={handleConsent}
|
||||
className="mt-3 w-full text-center text-xs text-stone-400 hover:text-stone-600 transition-colors">
|
||||
Use local AI anyway (not recommended for your device)
|
||||
</button>
|
||||
</div>
|
||||
);
|
||||
}
|
||||
|
||||
// Sufficient RAM: show the standard local AI onboarding.
|
||||
return (
|
||||
<div className="rounded-2xl border border-stone-200 bg-white p-8 shadow-soft animate-fade-up">
|
||||
<div className="flex flex-col items-center mb-5">
|
||||
@@ -68,6 +165,13 @@ const LocalAIStep = ({ onNext, onBack: _onBack, onDownloadError }: LocalAIStepPr
|
||||
</div>
|
||||
|
||||
<OnboardingNextButton label="Continue" onClick={handleConsent} />
|
||||
|
||||
<button
|
||||
type="button"
|
||||
onClick={handleSkip}
|
||||
className="mt-3 w-full text-center text-xs text-stone-400 hover:text-stone-600 transition-colors">
|
||||
Skip — use cloud AI instead
|
||||
</button>
|
||||
</div>
|
||||
);
|
||||
};
|
||||
|
||||
@@ -8,6 +8,27 @@ vi.mock('../../../../utils/localAiBootstrap', () => ({
|
||||
bootstrapLocalAiWithRecommendedPreset: vi.fn().mockResolvedValue({} as never),
|
||||
}));
|
||||
|
||||
vi.mock('../../../../utils/tauriCommands', () => ({
|
||||
openhumanLocalAiPresets: vi
|
||||
.fn()
|
||||
.mockResolvedValue({
|
||||
recommend_disabled: false,
|
||||
presets: [],
|
||||
recommended_tier: 'ram_2_4gb',
|
||||
current_tier: 'ram_2_4gb',
|
||||
selected_tier: null,
|
||||
device: {
|
||||
total_ram_bytes: 16 * 1024 * 1024 * 1024,
|
||||
cpu_count: 8,
|
||||
cpu_brand: 'test',
|
||||
os_name: 'test',
|
||||
os_version: '1.0',
|
||||
has_gpu: false,
|
||||
gpu_description: null,
|
||||
},
|
||||
} as never),
|
||||
}));
|
||||
|
||||
describe('LocalAIStep', () => {
|
||||
beforeEach(() => {
|
||||
vi.clearAllMocks();
|
||||
@@ -17,7 +38,8 @@ describe('LocalAIStep', () => {
|
||||
const onNext = vi.fn();
|
||||
renderWithProviders(<LocalAIStep onNext={onNext} />);
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: /continue/i }));
|
||||
const button = await screen.findByRole('button', { name: /continue/i });
|
||||
fireEvent.click(button);
|
||||
|
||||
expect(onNext).toHaveBeenCalledOnce();
|
||||
expect(onNext).toHaveBeenCalledWith({ consentGiven: true, downloadStarted: true });
|
||||
@@ -34,7 +56,8 @@ describe('LocalAIStep', () => {
|
||||
const onDownloadError = vi.fn();
|
||||
renderWithProviders(<LocalAIStep onNext={onNext} onDownloadError={onDownloadError} />);
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: /continue/i }));
|
||||
const button = await screen.findByRole('button', { name: /continue/i });
|
||||
fireEvent.click(button);
|
||||
|
||||
// onNext still fires immediately
|
||||
expect(onNext).toHaveBeenCalledOnce();
|
||||
@@ -53,7 +76,8 @@ describe('LocalAIStep', () => {
|
||||
const onNext = vi.fn();
|
||||
renderWithProviders(<LocalAIStep onNext={onNext} />);
|
||||
|
||||
fireEvent.click(screen.getByRole('button', { name: /continue/i }));
|
||||
const button = await screen.findByRole('button', { name: /continue/i });
|
||||
fireEvent.click(button);
|
||||
|
||||
expect(bootstrapLocalAiWithRecommendedPreset).toHaveBeenCalledOnce();
|
||||
expect(bootstrapLocalAiWithRecommendedPreset).toHaveBeenCalledWith(false, '[LocalAIStep]');
|
||||
@@ -67,11 +91,71 @@ describe('LocalAIStep', () => {
|
||||
const onNext = vi.fn();
|
||||
renderWithProviders(<LocalAIStep onNext={onNext} />);
|
||||
|
||||
const button = screen.getByRole('button', { name: /continue/i });
|
||||
const button = await screen.findByRole('button', { name: /continue/i });
|
||||
fireEvent.click(button);
|
||||
fireEvent.click(button);
|
||||
|
||||
expect(onNext).toHaveBeenCalledOnce();
|
||||
expect(bootstrapLocalAiWithRecommendedPreset).toHaveBeenCalledOnce();
|
||||
});
|
||||
|
||||
it('shows cloud fallback UI when device is below RAM floor', async () => {
|
||||
const { openhumanLocalAiPresets } = await import('../../../../utils/tauriCommands');
|
||||
vi.mocked(openhumanLocalAiPresets).mockResolvedValue({
|
||||
recommend_disabled: true,
|
||||
presets: [],
|
||||
recommended_tier: 'ram_2_4gb',
|
||||
current_tier: 'ram_2_4gb',
|
||||
selected_tier: null,
|
||||
device: {
|
||||
total_ram_bytes: 4 * 1024 * 1024 * 1024,
|
||||
cpu_count: 4,
|
||||
cpu_brand: 'test',
|
||||
os_name: 'test',
|
||||
os_version: '1.0',
|
||||
has_gpu: false,
|
||||
gpu_description: null,
|
||||
},
|
||||
} as never);
|
||||
|
||||
const onNext = vi.fn();
|
||||
renderWithProviders(<LocalAIStep onNext={onNext} />);
|
||||
|
||||
const cloudButton = await screen.findByRole('button', { name: /continue with cloud/i });
|
||||
expect(cloudButton).toBeTruthy();
|
||||
|
||||
fireEvent.click(cloudButton);
|
||||
expect(onNext).toHaveBeenCalledWith({ consentGiven: false, downloadStarted: false });
|
||||
});
|
||||
|
||||
it('allows force-enabling local AI on low-RAM device', async () => {
|
||||
const { openhumanLocalAiPresets } = await import('../../../../utils/tauriCommands');
|
||||
const { bootstrapLocalAiWithRecommendedPreset } =
|
||||
await import('../../../../utils/localAiBootstrap');
|
||||
vi.mocked(openhumanLocalAiPresets).mockResolvedValue({
|
||||
recommend_disabled: true,
|
||||
presets: [],
|
||||
recommended_tier: 'ram_2_4gb',
|
||||
current_tier: 'ram_2_4gb',
|
||||
selected_tier: null,
|
||||
device: {
|
||||
total_ram_bytes: 4 * 1024 * 1024 * 1024,
|
||||
cpu_count: 4,
|
||||
cpu_brand: 'test',
|
||||
os_name: 'test',
|
||||
os_version: '1.0',
|
||||
has_gpu: false,
|
||||
gpu_description: null,
|
||||
},
|
||||
} as never);
|
||||
|
||||
const onNext = vi.fn();
|
||||
renderWithProviders(<LocalAIStep onNext={onNext} />);
|
||||
|
||||
const forceButton = await screen.findByRole('button', { name: /use local ai anyway/i });
|
||||
fireEvent.click(forceButton);
|
||||
|
||||
expect(onNext).toHaveBeenCalledWith({ consentGiven: true, downloadStarted: true });
|
||||
expect(bootstrapLocalAiWithRecommendedPreset).toHaveBeenCalledOnce();
|
||||
});
|
||||
});
|
||||
|
||||
@@ -70,9 +70,15 @@ export const ensureRecommendedLocalAiPresetIfNeeded = async (
|
||||
return { presets, recommendedTier, selectedTier, hadSelectedTier: true, appliedTier: null };
|
||||
}
|
||||
|
||||
// No selected tier yet: persist the recommended tier so the Rust-side
|
||||
// `config_with_recommended_tier_if_unselected()` honors the user's
|
||||
// opt-in instead of defaulting a low-RAM device back to disabled.
|
||||
// The mount-time probe in LocalAIStep uses `openhumanLocalAiPresets()`
|
||||
// directly, so this apply only runs when the user has explicitly
|
||||
// chosen to proceed with local AI (consent flow).
|
||||
console.debug(
|
||||
`${logPrefix} applying recommended local AI preset`,
|
||||
JSON.stringify({ recommendedTier })
|
||||
JSON.stringify({ recommendedTier, recommendDisabled: presets.recommend_disabled ?? false })
|
||||
);
|
||||
await retryLocalAiCommand(
|
||||
'apply recommended local AI preset',
|
||||
|
||||
@@ -184,15 +184,20 @@ export interface PresetsResponse {
|
||||
current_tier: string;
|
||||
selected_tier?: string | null;
|
||||
device: DeviceProfileResult;
|
||||
/** When true the device is below the RAM floor and cloud fallback is the recommended default. */
|
||||
recommend_disabled?: boolean;
|
||||
/** Current value of `config.local_ai.enabled`. When false, cloud fallback is in use. */
|
||||
local_ai_enabled?: boolean;
|
||||
}
|
||||
|
||||
export interface ApplyPresetResult {
|
||||
applied_tier: string;
|
||||
chat_model_id: string;
|
||||
vision_model_id: string;
|
||||
embedding_model_id: string;
|
||||
quantization: string;
|
||||
chat_model_id?: string;
|
||||
vision_model_id?: string;
|
||||
embedding_model_id?: string;
|
||||
quantization?: string;
|
||||
vision_mode?: string;
|
||||
local_ai_enabled?: boolean;
|
||||
}
|
||||
|
||||
export interface LocalAiDiagnostics {
|
||||
|
||||
@@ -33,6 +33,24 @@ pub enum ModelTier {
|
||||
/// selection is re-enabled post-MVP.
|
||||
pub const MVP_MAX_TIER: ModelTier = ModelTier::Ram2To4Gb;
|
||||
|
||||
/// Minimum host RAM (in whole GB) below which the **default** is to skip
|
||||
/// local inference and use the cloud summarizer instead. The user can still
|
||||
/// override this and opt into local AI via settings.
|
||||
pub const MIN_RAM_GB_FOR_LOCAL_AI: u64 = 8;
|
||||
|
||||
/// Returns `true` when the device has enough RAM that local AI should be
|
||||
/// enabled by default. Below the floor we recommend cloud fallback instead.
|
||||
pub fn device_supports_local_ai(device: &DeviceProfile) -> bool {
|
||||
device.total_ram_gb() >= MIN_RAM_GB_FOR_LOCAL_AI
|
||||
}
|
||||
|
||||
/// Returns `true` when the device is below the RAM floor and local AI should
|
||||
/// default to disabled (cloud fallback). This is a **recommendation**, not a
|
||||
/// hard gate — the user can still opt in.
|
||||
pub fn should_default_to_cloud_fallback(device: &DeviceProfile) -> bool {
|
||||
!device_supports_local_ai(device)
|
||||
}
|
||||
|
||||
impl ModelTier {
|
||||
pub fn as_str(&self) -> &'static str {
|
||||
match self {
|
||||
@@ -178,12 +196,9 @@ pub fn preset_for_tier(tier: ModelTier) -> Option<ModelPreset> {
|
||||
}
|
||||
|
||||
/// Recommend a tier based on device capabilities.
|
||||
///
|
||||
/// The recommendation is capped at [`MVP_MAX_TIER`] so that auto-provisioning
|
||||
/// never selects a model above the MVP ceiling, regardless of available RAM.
|
||||
pub fn recommend_tier(device: &DeviceProfile) -> ModelTier {
|
||||
let ram_gb = device.total_ram_gb();
|
||||
let uncapped = if ram_gb >= 16 {
|
||||
let tier = if ram_gb >= 16 {
|
||||
ModelTier::Ram16PlusGb
|
||||
} else if ram_gb >= 8 {
|
||||
ModelTier::Ram8To16Gb
|
||||
@@ -194,17 +209,6 @@ pub fn recommend_tier(device: &DeviceProfile) -> ModelTier {
|
||||
} else {
|
||||
ModelTier::Ram1Gb
|
||||
};
|
||||
let tier = if uncapped.is_mvp_allowed() {
|
||||
uncapped
|
||||
} else {
|
||||
tracing::debug!(
|
||||
ram_gb,
|
||||
?uncapped,
|
||||
capped_to = ?MVP_MAX_TIER,
|
||||
"[local_ai] capping recommended tier to MVP ceiling"
|
||||
);
|
||||
MVP_MAX_TIER
|
||||
};
|
||||
tracing::debug!(ram_gb, ?tier, "[local_ai] recommended model tier");
|
||||
tier
|
||||
}
|
||||
@@ -318,13 +322,12 @@ mod tests {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn recommend_tier_by_ram_capped_at_mvp_max() {
|
||||
// All devices are capped at the single MVP tier (Ram2To4Gb).
|
||||
assert_eq!(recommend_tier(&test_device(1)), MVP_MAX_TIER);
|
||||
assert_eq!(recommend_tier(&test_device(3)), MVP_MAX_TIER);
|
||||
assert_eq!(recommend_tier(&test_device(4)), MVP_MAX_TIER);
|
||||
assert_eq!(recommend_tier(&test_device(8)), MVP_MAX_TIER);
|
||||
assert_eq!(recommend_tier(&test_device(32)), MVP_MAX_TIER);
|
||||
fn recommend_tier_scales_with_ram() {
|
||||
assert_eq!(recommend_tier(&test_device(1)), ModelTier::Ram1Gb);
|
||||
assert_eq!(recommend_tier(&test_device(3)), ModelTier::Ram2To4Gb);
|
||||
assert_eq!(recommend_tier(&test_device(4)), ModelTier::Ram4To8Gb);
|
||||
assert_eq!(recommend_tier(&test_device(8)), ModelTier::Ram8To16Gb);
|
||||
assert_eq!(recommend_tier(&test_device(32)), ModelTier::Ram16PlusGb);
|
||||
}
|
||||
|
||||
#[test]
|
||||
@@ -381,6 +384,25 @@ mod tests {
|
||||
assert_eq!(vision_mode_for_config(&config), VisionMode::Bundled);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn device_supports_local_ai_honors_min_ram_floor() {
|
||||
assert!(!device_supports_local_ai(&test_device(1)));
|
||||
assert!(!device_supports_local_ai(&test_device(4)));
|
||||
assert!(!device_supports_local_ai(&test_device(7)));
|
||||
assert!(device_supports_local_ai(&test_device(8)));
|
||||
assert!(device_supports_local_ai(&test_device(16)));
|
||||
assert!(device_supports_local_ai(&test_device(64)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn should_default_to_cloud_fallback_below_floor() {
|
||||
assert!(should_default_to_cloud_fallback(&test_device(1)));
|
||||
assert!(should_default_to_cloud_fallback(&test_device(4)));
|
||||
assert!(should_default_to_cloud_fallback(&test_device(7)));
|
||||
assert!(!should_default_to_cloud_fallback(&test_device(8)));
|
||||
assert!(!should_default_to_cloud_fallback(&test_device(16)));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn built_in_vision_modes_match_expectations() {
|
||||
let mut config = LocalAiConfig::default();
|
||||
|
||||
@@ -437,7 +437,11 @@ pub fn schemas(function: &str) -> ControllerSchema {
|
||||
inputs: vec![],
|
||||
outputs: vec![json_output(
|
||||
"presets",
|
||||
"Presets, recommended tier, current tier.",
|
||||
"Object containing: presets (array of ModelPreset), recommended_tier (string), \
|
||||
current_tier (string), selected_tier (string | null), device (DeviceProfile), \
|
||||
recommend_disabled (boolean — true when the device is below the RAM floor and \
|
||||
cloud fallback is the recommended default), local_ai_enabled (boolean — mirrors \
|
||||
config.local_ai.enabled so the UI can render the active state when disabled).",
|
||||
)],
|
||||
},
|
||||
"local_ai_apply_preset" => ControllerSchema {
|
||||
@@ -759,12 +763,16 @@ fn handle_local_ai_presets(_params: Map<String, Value>) -> ControllerFuture {
|
||||
preset_count = presets.len(),
|
||||
"[local_ai] presets: returning"
|
||||
);
|
||||
let recommend_disabled =
|
||||
crate::openhuman::local_ai::presets::should_default_to_cloud_fallback(&device);
|
||||
let value = serde_json::json!({
|
||||
"presets": presets,
|
||||
"recommended_tier": recommended,
|
||||
"current_tier": current,
|
||||
"selected_tier": selected_tier,
|
||||
"device": device,
|
||||
"recommend_disabled": recommend_disabled,
|
||||
"local_ai_enabled": config.local_ai.enabled,
|
||||
});
|
||||
Ok(value)
|
||||
})
|
||||
@@ -776,10 +784,26 @@ fn handle_local_ai_apply_preset(params: Map<String, Value>) -> ControllerFuture
|
||||
let tier_str = p.tier.trim().to_ascii_lowercase();
|
||||
tracing::debug!(tier = %tier_str, "[local_ai] apply_preset: parsing tier");
|
||||
|
||||
// Special "disabled" tier: turn local_ai off and route AI to cloud.
|
||||
if tier_str == "disabled" {
|
||||
let mut config = config_rpc::load_config_with_timeout().await?;
|
||||
config.local_ai.enabled = false;
|
||||
config.local_ai.selected_tier = Some("disabled".to_string());
|
||||
config
|
||||
.save()
|
||||
.await
|
||||
.map_err(|e| format!("save config: {e}"))?;
|
||||
tracing::debug!("[local_ai] apply_preset: local_ai disabled (cloud fallback)");
|
||||
return Ok(serde_json::json!({
|
||||
"applied_tier": "disabled",
|
||||
"local_ai_enabled": false,
|
||||
}));
|
||||
}
|
||||
|
||||
let tier = crate::openhuman::local_ai::presets::ModelTier::from_str_opt(&tier_str)
|
||||
.ok_or_else(|| {
|
||||
format!(
|
||||
"invalid tier '{}': expected one of ram_1gb, ram_2_4gb, ram_4_8gb, ram_8_16gb, ram_16_plus_gb",
|
||||
"invalid tier '{}': expected one of disabled, ram_1gb, ram_2_4gb, ram_4_8gb, ram_8_16gb, ram_16_plus_gb",
|
||||
tier_str
|
||||
)
|
||||
})?;
|
||||
@@ -788,19 +812,10 @@ fn handle_local_ai_apply_preset(params: Map<String, Value>) -> ControllerFuture
|
||||
return Err("cannot apply 'custom' tier; set model IDs directly".to_string());
|
||||
}
|
||||
|
||||
if !tier.is_mvp_allowed() {
|
||||
tracing::debug!(
|
||||
tier = %tier_str,
|
||||
"[local_ai] apply_preset: rejected — tier exceeds MVP ceiling"
|
||||
);
|
||||
return Err(format!(
|
||||
"model tier '{}' is not available in MVP; only tiers up to {} are allowed",
|
||||
tier_str,
|
||||
crate::openhuman::local_ai::presets::MVP_MAX_TIER.as_str()
|
||||
));
|
||||
}
|
||||
|
||||
let mut config = config_rpc::load_config_with_timeout().await?;
|
||||
// Re-enable local AI in case it was previously disabled via the
|
||||
// "disabled" tier, so the user can switch back to local inference.
|
||||
config.local_ai.enabled = true;
|
||||
crate::openhuman::local_ai::presets::apply_preset_to_config(&mut config.local_ai, tier);
|
||||
config
|
||||
.save()
|
||||
@@ -815,6 +830,7 @@ fn handle_local_ai_apply_preset(params: Map<String, Value>) -> ControllerFuture
|
||||
"embedding_model_id": config.local_ai.embedding_model_id,
|
||||
"quantization": config.local_ai.quantization,
|
||||
"vision_mode": crate::openhuman::local_ai::presets::vision_mode_for_config(&config.local_ai),
|
||||
"local_ai_enabled": true,
|
||||
}))
|
||||
})
|
||||
}
|
||||
|
||||
@@ -274,28 +274,31 @@ fn config_with_recommended_tier_if_unselected(config: &Config, device: &DevicePr
|
||||
let current_tier =
|
||||
crate::openhuman::local_ai::presets::current_tier_from_config(&config.local_ai);
|
||||
|
||||
// If a tier is already selected, check whether it exceeds the MVP ceiling.
|
||||
// If so, clamp it down to the recommended (MVP-capped) tier.
|
||||
// When the user has *not* explicitly picked a tier and the device is below
|
||||
// the RAM floor, default to disabled (cloud fallback). The user can still
|
||||
// opt in via Settings. This check runs first so even a "detected" tier from
|
||||
// matching defaults gets overridden on low-RAM hosts.
|
||||
if selected_tier.is_none()
|
||||
&& crate::openhuman::local_ai::presets::should_default_to_cloud_fallback(device)
|
||||
{
|
||||
tracing::debug!(
|
||||
total_ram_gb = device.total_ram_gb(),
|
||||
min_required_gb = crate::openhuman::local_ai::presets::MIN_RAM_GB_FOR_LOCAL_AI,
|
||||
"[local_ai] bootstrap: device below RAM floor, defaulting to disabled (cloud fallback)"
|
||||
);
|
||||
let mut effective_config = config.clone();
|
||||
effective_config.local_ai.enabled = false;
|
||||
return effective_config;
|
||||
}
|
||||
|
||||
// If the user has already picked a tier (explicit or matching defaults),
|
||||
// honor it. No MVP ceiling — users can pick any tier.
|
||||
if selected_tier.is_some()
|
||||
|| !matches!(
|
||||
current_tier,
|
||||
crate::openhuman::local_ai::presets::ModelTier::Custom
|
||||
)
|
||||
{
|
||||
if !current_tier.is_mvp_allowed() {
|
||||
let recommended = crate::openhuman::local_ai::presets::recommend_tier(device);
|
||||
tracing::warn!(
|
||||
?current_tier,
|
||||
?recommended,
|
||||
"[local_ai] bootstrap: selected tier exceeds MVP ceiling, clamping to recommended"
|
||||
);
|
||||
let mut effective_config = config.clone();
|
||||
crate::openhuman::local_ai::presets::apply_preset_to_config(
|
||||
&mut effective_config.local_ai,
|
||||
recommended,
|
||||
);
|
||||
return effective_config;
|
||||
}
|
||||
return config.clone();
|
||||
}
|
||||
|
||||
@@ -360,51 +363,88 @@ mod tests {
|
||||
assert!(!service.should_run_memory_autosummary(&config));
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bootstrap_clamps_non_mvp_tier_when_selection_missing() {
|
||||
let config = Config::default();
|
||||
// Default config matches Ram8To16Gb which is not MVP-allowed,
|
||||
// so bootstrap clamps it to the MVP-capped recommended tier.
|
||||
let device = DeviceProfile {
|
||||
total_ram_bytes: 4 * 1024 * 1024 * 1024,
|
||||
fn test_device(ram_gb: u64) -> DeviceProfile {
|
||||
DeviceProfile {
|
||||
total_ram_bytes: ram_gb * 1024 * 1024 * 1024,
|
||||
cpu_count: 4,
|
||||
cpu_brand: String::new(),
|
||||
os_name: String::new(),
|
||||
os_version: String::new(),
|
||||
has_gpu: false,
|
||||
gpu_description: None,
|
||||
};
|
||||
|
||||
let effective = config_with_recommended_tier_if_unselected(&config, &device);
|
||||
|
||||
assert_eq!(
|
||||
effective.local_ai.selected_tier.as_deref(),
|
||||
Some("ram_2_4gb")
|
||||
);
|
||||
assert_eq!(effective.local_ai.chat_model_id, "gemma3:1b-it-qat");
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bootstrap_clamps_non_mvp_selected_tier() {
|
||||
fn bootstrap_defaults_to_disabled_on_low_ram_device() {
|
||||
let config = Config::default();
|
||||
let device = test_device(4);
|
||||
|
||||
let effective = config_with_recommended_tier_if_unselected(&config, &device);
|
||||
|
||||
assert!(
|
||||
!effective.local_ai.enabled,
|
||||
"local_ai.enabled must default to false on <8 GB device"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bootstrap_keeps_enabled_on_sufficient_ram_device() {
|
||||
let config = Config::default();
|
||||
let device = test_device(16);
|
||||
|
||||
let effective = config_with_recommended_tier_if_unselected(&config, &device);
|
||||
|
||||
assert!(
|
||||
effective.local_ai.enabled,
|
||||
"local_ai.enabled must stay true on >= 8 GB device"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bootstrap_respects_explicit_tier_on_low_ram_device() {
|
||||
let mut config = Config::default();
|
||||
config.local_ai.selected_tier = Some("ram_2_4gb".to_string());
|
||||
crate::openhuman::local_ai::presets::apply_preset_to_config(
|
||||
&mut config.local_ai,
|
||||
crate::openhuman::local_ai::presets::ModelTier::Ram2To4Gb,
|
||||
);
|
||||
let device = test_device(4);
|
||||
|
||||
let effective = config_with_recommended_tier_if_unselected(&config, &device);
|
||||
|
||||
// User explicitly chose a tier, so local_ai stays enabled.
|
||||
assert!(
|
||||
effective.local_ai.enabled,
|
||||
"explicit tier selection must be honored even on low-RAM device"
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bootstrap_keeps_default_tier_when_selection_missing_and_ram_sufficient() {
|
||||
// Default config already matches a preset (Ram8To16Gb) and device has
|
||||
// enough RAM, so bootstrap returns it untouched (no auto-change).
|
||||
let config = Config::default();
|
||||
let device = test_device(8);
|
||||
|
||||
let effective = config_with_recommended_tier_if_unselected(&config, &device);
|
||||
|
||||
assert!(effective.local_ai.enabled);
|
||||
assert_eq!(
|
||||
effective.local_ai.chat_model_id,
|
||||
config.local_ai.chat_model_id
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn bootstrap_honors_explicit_non_mvp_tier_selection() {
|
||||
let mut config = Config::default();
|
||||
config.local_ai.selected_tier = Some("high".to_string());
|
||||
let device = DeviceProfile {
|
||||
total_ram_bytes: 4 * 1024 * 1024 * 1024,
|
||||
cpu_count: 4,
|
||||
cpu_brand: String::new(),
|
||||
os_name: String::new(),
|
||||
os_version: String::new(),
|
||||
has_gpu: false,
|
||||
gpu_description: None,
|
||||
};
|
||||
let device = test_device(8);
|
||||
|
||||
let effective = config_with_recommended_tier_if_unselected(&config, &device);
|
||||
|
||||
// "high" maps to Ram16PlusGb which is not MVP-allowed, so it gets clamped.
|
||||
assert_eq!(
|
||||
effective.local_ai.selected_tier.as_deref(),
|
||||
Some("ram_2_4gb")
|
||||
);
|
||||
assert_eq!(effective.local_ai.chat_model_id, "gemma3:1b-it-qat");
|
||||
// User explicitly picked a high-tier preset — honor it, no MVP clamp.
|
||||
assert_eq!(effective.local_ai.selected_tier.as_deref(), Some("high"));
|
||||
}
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user