thoughts.txt [慧慧]

recently my dad shared the term "silver-haired economics" with me: markets built around our beloved elders.

'By 2035, more than 400 million Chinese citizens will be aged 60 or above. The Fudan Institute on Aging estimates that the silver economy could reach 19.1 trillion yuan (approximately USD 2.7 trillion), accounting for nearly 28% of national consumption.'

i feel like in the US there's a lot of obssession to be at "the frontier". and yet china, whilst also there, also builds so much at the periphery -- electric wheelchairs (i saw a few in sz and they look so sleek!), granfluencers on tiktok growing a following, wearables for remote health tracking, robotic guiding dogs, smart beds, exoskeletons for walking, etc.

many year in reviews, and more to come. my own thoughts this year:

• ai application layer: buzz word of all future predictions for 2026. not going to elaborate more, but refer to [1], [2], [4], [5], [8]. they need to be future-proof (advancements in models will improve the product and not replace it), have strong context management, and be proactive.
• on the edge: our data is in everything. sapped from our attention and clicks and stored in some cloud. i think people will care about reclaiming their data. that, with SLMs and local models, i see on-device AI-native phones and products being the future. @apple, where you at?
• off the grid: people are tired. merriam webster word of the year -- slop. that's when you know it's bad. people want brick phones. people want long attention spans and instagram without reels and real connection. it's going to be a year of (ironically) influencers in meditation camps, touching grass, eco villages, long hikes. i think that's why i like something like plaud -- yes it's an ai product, but it's for irl convos.
• world models and robotics : jagged intelligence (andrej coins in [2]), and then ofc google deepmind's work [8], and then china's big robotic advancements in [9].
• resistent minds, inert bodies: on x, online, at work, the folks around me lock in hard on AI. they're up to date with the latest advancements and tech. on the ground -- the auntie at a cha chaan teng, bus driver, C-suite in big firms -- less willing to adopt. we're creating apps that challenge an ingrained workflow of 20 years -- to some degree, that's also why microsoft copilot struggles (other than it being not very good haha). you have accountants and consultants who breathe excel, set every shortcut, kiss every key with their grubby fingers every day... telling them to use AI chat to help them by writing their formulas wrong, mess up their tables... no thank you. adoption will take time and require a cultural shift of some sort, but it's possible and slowly happening. think MIT report.

safe to say that anyone not living in china is often surprised when they visit to see the sheer technological and social advancements of the city. see: all the x posts from visitors, research paper decoupling (chinese arvix), products that are very 普遍 without an exact "international twin" (e.g. 小天才). this is me included -- whenever i visit, i often find something "new" and my mum would be like "this has been here the past year now".

a part of this reminds me of the macartney embassy to china in 1793, during the late qing dynasty -- bringing in watches and clocks that the dynasty was neither surprised by nor interested in (tho this was right before the industrial revolution and opium wars. life would have been very different if they established diplomatic relations). there's a history of international states being blind to / underestimating china. ofc a big part of it are the protectionist policies + decoupling + china-us relations and media portrayals disincentivising foreign visitors.

this month i visited zhangjiajie, zhuhai, and frequent shenzhen. one main observation (from my narrow point of view) is that tech is for everyone and surfaces in different forms. i made a list of "unassuming tech" that ive observed during my few visits:

• rideshare pickup with computer vision: detecting car plates and where each car is pulling up, shown on multipe screens so passengers know where to go (zhuhai)
• cleaning bots: on roads, malls, streets, train stations -- it feels like a natural part of the city, almost so much so that i dont take note of it (shenzhen).
• exoskeleton legs for hiking and elderly: tried this after hiking for 5 hours on a hurt ankle, and wow i felt light as a feather. it's meant for elderly and hurt, but they beta released this to big hiking spots in the country (zhangjiajie).
• drink fridges (not just vending machines): where you can unlock with your phone, grab your drink, and leave -- it auto-detects what drinks youve taken (zhangjiajie)
• limited smartwatches for kids (xiao tian cai): noticed a lot of kids wearing these -- it has location tracking, can create communication channels with two-way consented individuals (e.g. parents), stripped down version of alipay, little camera (saw twins taking photos of trees with it!)
• facial recognition, phone integration, and ID linkage: every ticket into a tourist location linked to your identity, cameras at danger spots (e.g. 10km into a hike, or at a hard crossing in via ferrata), entering spots with only your face (zhangjiajie). side note -- toilets everywhere!
• niche machines: ceramic pot to roast sweet potatoes with a thermometer outside, and racks lining the side. sugarcane chopper machine. there's a product for everything (and can be found on taobao for pretty cheap!)
• and then ofc LLMs!: angry tourist on the train speaking to a worker, uses LLM (i think doubao from over the shoulder peeping) for translation and assistance.

im sure youve felt it too -- talking to any LLM, responses tend to be quite generic, no?

"Artificial Hivemind: The Open-Ended Homogeneity of Language Models (and Beyond)". this paper systematicised the evaluation of all the models to show that model results are converging (same prompt -> very similar responses)

• new dataset, "INFINITY-CHAT": a large-scale dataset of 26K real-world open-ended queries spanning diverse, naturally occurring prompts
• the Artificial Hivemind effect: (1) intra-model repetition, where a single model repeatedly generates similar outputs, and, more critically, (2) inter-model homogeneity, where different models independently converge on similar ideas with minor variations in phrasing. this is not just a matter of LLM "tuning" to the dataset, but a more fundamental property of the models.
• what's causing this? training on synthetic data, insufficient diversity in training data

implications:

• model selection: if it's true that we are seeing some sort of ensembling / collapsing of model choices, it may imply it matters less to base model picks on its "general quality of responses" (obv there are still differentiators like speed, model size, use case and context, tone etc.)
• edge computing, specialised use cases: yes these foundational AI companies want "AGI", "general" intelligence -- but for industry applications imo, folks want specialised models (e.g. healthcare keywords, SWE trained models). to me it sounds like opportunity to train context-specific models, even your own mini-local one with your own data.
• lack of creativity, bias, responsible ai, etc.: all encoding similar biases, all representing a majority voice, all thinking the same way. cliches from head to tail.

will we reach agi or is it an illusion? on x i float between the agi-bulls and agi-bears, and convos stretch from the most abstract and philosophical (e.g. what even is consciousness? what is human?) to technical (e.g. scaling laws, GPUs, computation). reading just a few hairs off of the wide scalp of the space, here are a few strands of thoughts.

definition and scope of "general intelligence": what is AGI? think this as the philosophical backbone -- defining human consciousness, intelligence, human distinctiveness. themes such as materialism, anthropomorphism, researching other forms of consciousness such as animal. what are the capabilities and behavious agi should have? as PMs like to say: what is the definition of done?

• moving the goalposts: first AI was defined as being able to perform functional tasks, like computing numbers,but now we're asking for reasoning, introspecting, learning
• tendency of the human to anthropomorphise: intelligence doesn't have to look like us. but also when talking to LLMs that are essentially next-token prediction, folks interact with it like a fellow human.
• david chalmer's hard problem: brain processes and how they surface first-person experience.
• emergence as a challenge to materialism: complex systems can behave in surprising and exceptional ways that we have yet to understand (e.g. the mind), yet not-understanding doesn't dispute with the fact that we consider ourselves "intelligent"
• richard dawkins: "the illusion that we are a unit and not a colony”

building the "artificial": how do we get to agi? convos around scaling laws, compute, computational representations of the mind. e.g. LLMs, agentic architectures, world models and embodied AI. memory systems, continual learning, self-reflection. exploring how agi can surface.

• identifying constraints: based on 1 and 2, what can we do now/soon? can GPUs/TPUs keep up? do we have enough data (e.g. real world robots), are our algos efficient enough? this gets more into the mathematical and technical.
• scaling laws vs downsizing: yes we can scale more, make compute more efficient, but also we're limited by energy, data availability, budget etc. for those in the downsizing camp, deciding how to downsize is the challenge -- data quality? what sources? how to implement a pruning / forgeting mechanism?
• current frontier model limitations: the "jagged" edge of LLM performance. yes it can solve math olympiad problems, but it can't set a table. it doesn't extrapolate well nor understand "basic human things", e.g. object permanence and continuity. a lot of what it can do it has to learn ("fine-tune"). also, lacks continual learning.

it's true that language encodes many forms of our intelligence. and it's quite magical. think about it: words need to be boundless but also structured. language is a formulation of syntax and semantics ("colourless green ideas sleep furiously"), embued with micro-worlds and meaning only through human interaction and context. a lot of what is perceived to distinguish humans from other animals is intimiately bound with our linguistic abilities.

but what i think we often mistaken when interacting with LLMs, is that language = intelligence. language presents a tool for us to represent and communicate our worlds to one another. judith fan gives a good example -- describe a specific bookshelf to another person.

• angle 1: "it's a bookshelf"
• angle 2: "it's a place where you can store books"
• angle 3: "it has 8 cubic holes for storing reading material"
• angle 4: "30x30x30cm wooden planks, multiple that by 8 times, place them together in a rectangular shape, store these items that have many 30x50cm pages, all composed of words into binders...etc, etc."

language layers abstraction over abstraction, formatting our experiences in a way that's both efficient and mutually understood between two people.

so, while yes, language encodes 'core intelligence' in a sense, it is not purely intelligence itself. words are spoken and heard and thought, but what we're lacking include the other sensory inputs of the world: sight, touch, and even time -- evolution and continual learning and building of linguistical blocks on how we got from line to bookshelf. this bleeds into fei-fei li and yann lecun's work on world models. but that's for another day.

3-part series on "society of the psyop" by Trevor Paglen::

there will always exist a gap between "objective reality" and our phenomenological experience -- the qualia, shaped by memories, expectations, upbringing, culture, belief systems, etc. the article argues that this is an exploitable gap, a gap that "can be filled with all sorts of prompt injections and adversarial hallucinations". think propaganda, disinformation, conspiracy. think media manipulation, deepfakes, political polarisation from the crumbling of facts. "desire to believe eclipses the evidence at hand".

when talking to LLMs, i no longer bother to write in full sentences, nor do i even attempt to express myself artically. i dont even fix my typos. even before i get stuck it spoonfeeds me suggestions on the next possible word. more and more, reading non-LLM articles feel exhausting and my attention careens.

but i can tell. i cant tell if an article is necessarily LLM-generated, but i can tell when it's not.

for the past decades, literacy rates were a part of measuring a country's level of development. in the LLM era, why do words still matter?

• "Humans achieve agency by composing their lives in their own language.": LLMs produce words, but not meaning. meaning comes when the words are your own, able to produce new experiences and ideas and "activates fresh existence" when spoken to someone else, to yourself, to some god. words form narrative and empowerment. to expand your vocab with new imagery, metaphors, is to allow you to explicate how you feel, what you think and believe more precisely. you build relationships with your words and they form a personal understanding with you. it's like when we used to learn a new SAT word and would stuff it into sentences. inorganic at first, but eventually you'd gain more nuance on how the word fits into your expression.
• words as historical artifacts: words aren't just words, they are a history. as someone who grew up in hong kong speaking english as my first language, i recognise my relationship with language stems from a colonial legacy, a british empire. with LLMs, history is conflated into a monolithic archive of training data. as Alain Mabanckou asks: "AI may well be able to take on board these cultural elements, but can it also reproduce the suffering of these oppressed peoples?"
• beauty, resonance, aesthetic: i write poetry. a part of poetry is surprise, to draw an unfamiliar connection between two things (within the constraint of what 'makes sense', however that's defined, but even this can be challenged e.g. by surrealists) -- the churning washing machine and the cycle of poverty, the moon and a belly button. LLMs predict predictable predictions. the next most probable token. words are not just tools, but things that draw beauty.

we've been trying to solve memory in neural architectures since the 50s, starting from symbolic memory (SOAR architecture) to RNNs and LSTMs with gates to control information flow, then the legendary "Attention is all you need" paper in 2017 introducing a context window as memory (-> context rot, lost in the middle), and now finally external memory banks (RAG, graphs, scratch pads). catastrophic forgetting continues. labs rush to solve it.

• memory is not a bag of facts, it's layered control: it's not just about storing information, it's about being selective about what to remember and what to forget. pruning and forgetting are equally important when it comes to memory
• memory != continual learning: learning isnt just remembering and is never static ofc. LLMs are static post-pretraining and limited to in-context learning, lacking the ability for adaptation and "neuroplasticity"
• formation of long-term memory involves two consolidation processes: online (synaptic) consolidation soon after learning where new info is stabilised and being transferred from ST to LT storage, then offline (systems) consolidation, replaying recently encoded patterns with sharp-wave ripples in the hippocampus

i enjoyed this article whilst having my morning coffee:

• 'Fragility, and the culture it creates, can be an asset in inspiring the sort of care necessary for the long term. A system that seeks the indestructible or infallible has the potential to encourage overconfidence, nonchalance, and the ultimate enemy of all archives, neglect.'
• 'The success of century-scale storage comes down to the same thing that storage and preservation of any duration does: maintenance. The everyday work of a human being caring for something. [...] How it is stored will evolve or change as it is maintained, but if there are maintainers, it will persist.'

as someone who likes to document everything, but rarely goes back to read it, i often question what the value of preservation is. search requires you to remember what to search for. scrolling requires time. all the bits of myself floating in the digital void. id be curious to fine-tune an LLM with all my journal entries, notes, images, thoughts, to see whether it can evolve into a mini-me.

i first heard the phrase "china-us decoupling" as a junior at berkeley [1]. that was 6 years ago. since then, the phrase has been tossed by media outlets and gnawed on by economists and friends [2,3 to show a few examples]. i never thought too much about it, but recently moving between hk/shenzhen and US, i feel like i have to adopt a new set of lingo, learn two separate UI flows, download two sets of apps, with each place:

• seeing double in my homescreen: consumer apps like google maps vs gaode (alibaba), whatsapp/messenger vs wechat, foundational AI (deepseek, kimi, doubao, openai, anthropic), electric cars and home products (biyadi, xiaomi, xiaopeng, weilai, lixiang, vs tesla/lucid). last year i came home and the roads were packed with teslas, now i see more biyadis. used to using google maps, the gaode flow felt confusing and awkward to me. i learnt to buy meituan coupons for discounts at restaurants before checking out, and topping up my weixin/alipay wallet.
• consumer products: i went to decathalon in shenzhen to buy protein bars and couldn’t find PhD, barbells, or any international brand, just local Chinese ones.
• twin markets finding parity: talking to my dad this morning, he shared alibaba's origin story, starting from replacing yellow books as a registrar, to becoming an ecommerce marketplace that started the same way amazon did, as a bookstore. on the flip side, saw on x that livestream sales in clothing stores beginning to grow in US, a trend started in china. two separate markets learning from each other.
• data decoupling, the beginnings: alibaba is growing its cloud business, competing against AWS. many multinationals in china still use AWS, but with the decoupling of foundational LLM models and their training data, will we move up the chain to even decouple all that we store in the cloud?

massive traffic, and my dad chuckles to himself and says "must be a self driving car glitching out up front". he turns to my mum: "with the way you drive, maybe it's trained on your data".

thinking about the "long tail" of edge cases that self driving cars need to be trained on and the success it can have in china:

• crazy drivers, unlearnable behaviour? with the way folks drive in shenzhen + traffic rules (more likely not followed than followed), it feels too "unpredictable" for an automated system to navigate
• protecting workers: even if successful, the government may not be extremely open to mass deployment as it takes economic jobs in an already high unemployment market. [3, 5, 6]
• chinese autonomous vehicle market: with baidu's apollo go, weride, pony ai, all in testing phase but on international roads: singapore, abu dhabi, dubai. why not chinese roads? too difficult to learn, or additional regulations? [2]

other robot things:

• humanoid vs non-humanoid: make humanoid robots as most of society is shaped by the human form. but it's inefficient. why build a humanoid robot to drive a car when you can just build a self-driving car?
• imo china will just dominate. anything that requires hardware and materials, china just has such a strong competitive advantage with the manufacturing base.

foundational models are slowing down, the ai product space is beginning to grow. there's a lot of noise, a lot of "chatgpt" wrappers, 20 YC companies doing a similar thing: medical advice ai, loveable for x, palantir for y, etc. yet not one has stood out yet. we're in a space of high-competition. i see so many twitter posts of new company launched announcements, but then never hear about them again. what will stick?

• right now: they lack trust loops and they don't solve real pain points, sure there is a novelty but feels like people try them once then leave
• ai as a feature, not a destination: companies that already embed customers into their workflows, e.g. notion, figma, salesforce, will win by absorbing ai into existing user behaviour rather than asking users to switch tools. if we built an ai product as standalone it either has to be 1) revolutionary for a current workflow; 2) currently unsolved / manual in a niche, narrow market
• unique data as competitive advantage: e.g. an ai coach who knows your habits knows your preferences, progressions, and can tailor education to you;
• high-trust verticals: domains that like reliability, accuracy, emotional intelligence, the human touch, e.g. law and healthcare may adopt slower
• how much do people actually use AI? am i just in a bubble??

the internet allowed for massive decentralisation of voices, opinions, perspectives, but also infinite connection for folks using network effects. you can find blogs with 1 view, videos from the 2000s of a skater kid, tumblr journal entries, recipes half written... fragmented by design where the long tail thrived. genAI is now doing the opposite. no one goes to google for search, they go to chatgpt for their query and chatgpt uses the "top 10 relevant results", summarises and blends the individual authors' voices into the murky brown of mixed paint, and provides you the result. it abstracts and regurgitates the mess. likely you won't fact check it. likely you won't click into the sources. what does this mean for us? what does this mean for the internet? the internet is no longer a place for us to connect, and voices of the "influential", the ones with higher SOE, higher views, to get control. i mean in a way the internet is a reflection of society.

• openai, anthropic beginning to create these product platforms: e.g. you can book hotels directly in chat, you can make a list of shopping options, plan your tour. pure productivity optimisation, convenience brainrot, let us do this for you, offload your cognition, full personal assistant, or just a person, you?
• Kierkegaard warns: "eventually human speech will become just like the public: pure abstraction—there will no longer be someone who speaks, but an objective reflection will gradually deposit a kind of atmosphere."
• smooth communication, texture of human communication gets lost, the typos, tangents, style, what tech folks like to call "taste"
• return of the retro: unoptimised spaces, in-person interactions, grassroots community building. think the old myspace, or facebook in 2010s where we posted on one other's walls. strava-style connection is what we're looking for: small circles, close friends, un-curated, authentic

you know somedays i realise my brain is truly empty and there is nothing there, because i dont give it the space to breathe. i execute ny 9-6, exercise, eat, sleep, and recycle. im scared to be bored, and because of this i fill free time with noise -- another youtube documentary, podcast, the occasional insta reels void. this is my accountability tracker. no fancy designs, just raw text.

hello world, i live.