Dunbar goods

Subset's mission is to usher in a new way to save, share and search. This quest is driven by the realisation that saving, sharing and searching are intricately connected behaviours. The routes they etch in the cognition of an agent—be it an individual or an institution—influence and steer one another in complex ways.

To start, we've focused on how we save and share. These are the more pressing problems with more immediate resolutions. Search, however, has not been overlooked. We've been asking ourselves a simple question: "What is it about the modern search experience that is so peculiarly lacking?"

Unsurprisingly, there are multiple factors at play. Platform battlement raising; the "enshittification" of the web; increasingly explicit user activity monetisation and shady data dealing; the internal incentive structures of the dominant search providers; the principal-agent problems that play out on the market-side of search; the seeming inadequacy of AI tech for fine grained discovery and retrieval; the retreat of users to the cozy web—these are all meaningful contributors to the bleugh-ness of modern search. Yet these issues obscure a fundamental inadequacy.

Today, a particular class of web citizen remains unrecognised, and the goods they produce remain illegible. This post focuses on the latter part of this fundamental inadequacy: Dunbar goods and their resilience to contemporary search paradigms and communication practices.

A Dunbar good is a composite assemblage of information whose value is legible only to a small, difficult-to-define group. Named in honour of the famous Dunbar number, these goods are accessible primarily through immersion or proximity to a specific group, community or locale. Within this collective, Dunbar goods are readily available and regularly used.

The impact of Dunbar goods is multi-faceted and often elusive. They support local innovation, protect against knowledge loss, facilitate knowledge transfer and serve as focal points around which members of the collective cohere. Typically originating from influential or experienced figures within the group, these goods aren't owned or considered proprietary and are shared informally and ad hoc.

Sustained through in-group usage and real-world relevance, Dunbar goods are culturally bound to the collective and ensnared within in-group norms and shibboleths. To outsiders, these goods are challenging to detect, classify or value, often appearing as guarded secrets or niche hacks when they're surfaced.

Consequently, outsiders might refactor Dunbar goods as either private goods insufficiently leveraged or public goods deserving wider dissemination. However, upon such attempts, it becomes clear that Dunbar goods lack easily extractable value and wider appeal.

Examples of Dunbar goods include:

• Lists of resources compiled by industry practitioners and experts

• Insight shared through mentorship programs or community workshops

• Informal knowledge networks, like meet-ups, societies and group chats

• Group- or domain-specific best practices shared in professional contexts

Understanding the resilience of Dunbar goods to contemporary search paradigms and communication practices requires a quick historical detour. So let's do a rapid survey of the eras of communication and the paradigms of search.

The history of communication and information transfer can be segmented into distinct eras, each defined by its technological advancements and societal practices.

The oral tradition era spans from prehistoric times to the advent of writing systems. It's characterised by the transmission of information through oral traditions, stories, and songs within small, localised communities.

This decentralised method persisted until around 3000 BCE, when writing systems like cuneiform and hieroglyphs emerged, heralding the written word era. During this phase, information became more centralised, with scribes and elites controlling its recording and dissemination, though its scope remained limited to specific civilisations such as Sumer, Egypt, and China.

The manuscript era followed, stretching from the early Middle Ages to the invention of the printing press in the 15th century. During this period, religious and academic institutions centralised information by meticulously transcribing texts. The invention of the printing press marked the beginning of the print era, which facilitated the broader dissemination of literature, news, and scientific discoveries on a national and eventually global scale.

The broadcast era of the early 20th century saw the rise of radio and television, extending the reach of information while centralising content production within major broadcasting corporations.

The digital and internet era, starting in the late 20th century, brought about instantaneous global information exchange, with a mix of centralised control by major tech companies and decentralised content creation through blogs, forums, and social media.

Currently, the emerging post-Internet and distributed ledger era points toward increased decentralisation, leveraging technologies for peer-to-peer information sharing without centralised intermediaries, often via the means of protocols.

The evolution of search paradigms reveals significant technological and methodological advancements over time.

We begin with the manual/literal search paradigm, which spans from ancient times to the early 20th century. During this era, individuals relied on physical texts and archives, often necessitating direct access to repositories like libraries. This decentralised method limited accessibility by physical location and heavily depended on human skill and accuracy.

The advent of card catalogues and indexing systems in the 18th century marked a shift towards centralised systems maintained by libraries and institutions, utilising methods like the Dewey Decimal Classification to improve search efficiency and organisation. In the 1960s, the early computational/database search paradigms emerged, centralising searches through database management systems and Boolean search queries, expanding the scope from national to international.

The internet directory-based search paradigm of the early 1990s to late 1990s allowed for global reach through curated directories like Yahoo! Directory, though human curation limited scalability. The subsequent early search engine paradigm, from the mid-1990s to early 2000s, saw further centralisation with companies like AltaVista and early Google using basic keyword searches and page-ranking algorithms.

Entering the 21st century, the modern algorithmic search paradigm emerged, driven by advanced algorithms, machine learning, and data-driven personalisation, and dominated by tech giants such as Google and Microsoft. The social and collaborative search paradigm, starting in the late 2000s, integrated user-generated content and crowdsourced data from both centralised and decentralised platforms like social media and Wikipedia.

In the 2010s, the contextual and conversational search paradigm came to the forefront, with natural language processing and voice-activated assistants like Google Assistant and Alexa offering immediate responses and multi-turn dialogue systems. Alongside this, the semantic and knowledge graph search paradigm began to leverage technologies to understand relationships and entities through knowledge graphs, driven by innovations from companies like Google and IBM.

Finally, we are now seeing the emergence of the decentralised/search-less paradigm in the 2020s, characterised by blockchain and peer-to-peer technologies, focusing on privacy, security, and user-controlled data, and moving away from traditional centralised models.

Consider the character, the shape and texture of Dunbar goods alongside the different eras of communication and search paradigms. What is it about Dunbar goods that guarantees their resilience to modern communication methods and their inaccessibility to contemporary search?

First, although Dunbar goods can be represented and transferred digitally, the dynamics of their replication and passage mimic the oral communication era. At the most local scales, these goods are transferred quickly, accurately and effectively. However, attempts to communicate them beyond this scale result in a non-linear degradation at best, and at worst an increasing distortion and fundamental poisoning of the goods.

Second, because Dunbar goods are informational assemblages—highly contextual compilations of related but distinct units or informational goods—they are notoriously difficult to index and categorise. Imagine a simple sort algorithm that splits an array of values recursively until only singular items remain, then performs sequential pairwise comparisons to return a sorted array. Splitting a Dunbar good dissolves its value, and this value is not even comparable to anything outside the group or collective context where it emerged.

Finally, both the different eras of communication and search paradigms prioritise exchange. Much of the mechanisms, the methods and the outcomes have been orientated towards commerce. Dunbar goods, however, defy exchange. They do not fit within its boundaries, as they are children of the gift paradigm—things given and shared without the confines of strict and immediate reciprocity aimed at currency-recognised outcomes.

As you may have guessed, Subset intends to envelop Dunbar goods within the new way of saving, sharing and search we're developing. The exact mechanics of how we will accomplish this remains an open question, but our goal is clear: make Dunbar goods easy to save, simple to share, and fast to search. Critically, we must do this in a way that:

• Honours and respects the gift paradigm within which these goods emerged

• Preserves their localised value whilst making it discoverable

• Sustains the oral-like dynamics of their communication

If we can accomplish this, then search becomes what we all want it to be—not just a tool for finding the old but a way to generate the new. An enlivening experience instead of a deadening one.