It’ll be “local AI” when they let me point to my own self hosted inference servers, rather than simply OpenAI, Anthropic, or Gemini as providers. Soon to include Apple provider, I guess.
I’ve used the Apple Intelligence ecosystem. The models are slightly acceptable in extremely small context windows, but they completely shit the bed for any kind of practical ad-hoc use. Even if you try to dumb it down to like 6 words. It’s trash. It couldn’t even find a picture of my finger with the keyword “finger.” It couldn’t explain basic details of my phone… it was like interacting with a shittier version of ChatGPTs first release — much shittier.
It told me I have an iPhone 18. I have a 17 Max Pro, the 18 hasn’t been released yet. I don’t plan to ever buy the 18, given the hardware regression with their charging port. But… as I said, it’s not even released yet.
That’s fine with me, actually. If that’s all my phone can handle then so be it. But, when I eventually and obviously will want something more practical, my only options shouldn’t be to pay frontier cloud models if I want deep integration with my phone. The only alternative shouldn’t be to subscribe to a higher tier iCloud+.
I can put a vpn on my phone to access vLLM or ollama locally. Why won’t they let me use that?
It’ll be “local AI” when they let me point to my own self hosted inference servers, rather than simply OpenAI, Anthropic, or Gemini as providers. Soon to include Apple provider, I guess.
I’ve used the Apple Intelligence ecosystem. The models are slightly acceptable in extremely small context windows, but they completely shit the bed for any kind of practical ad-hoc use. Even if you try to dumb it down to like 6 words. It’s trash. It couldn’t even find a picture of my finger with the keyword “finger.” It couldn’t explain basic details of my phone… it was like interacting with a shittier version of ChatGPTs first release — much shittier.
It told me I have an iPhone 18. I have a 17 Max Pro, the 18 hasn’t been released yet. I don’t plan to ever buy the 18, given the hardware regression with their charging port. But… as I said, it’s not even released yet.
That’s fine with me, actually. If that’s all my phone can handle then so be it. But, when I eventually and obviously will want something more practical, my only options shouldn’t be to pay frontier cloud models if I want deep integration with my phone. The only alternative shouldn’t be to subscribe to a higher tier iCloud+.
I can put a vpn on my phone to access vLLM or ollama locally. Why won’t they let me use that?
On android you can just use something like jegly Box and just run Gemma 4 or some other model. Decent results for an offline ai.