From Transformers to NEAR: The Journey of Illia Polosukhin

Illia Polosukhin has always been a prominent figure in the realm of techonolgy. In 2014, he joined Google Research, where he worked on developing the models and tools that would eventually become the pillars of modern Artificial Intelligence, and specifically generative AI. As a member of the team that published the landmark AI paper “Attention Is All You Need”, he is one of the co-creators of the transformer model which has revolutionized the field of natural language processing.

The transformer model is been using widely to power generative systems like ChatGPT (the ‘T’ stands for transformer). The transformer model also established a new paradigm in generative AI. Its self-attention mechanism allows the model to generate highly coherent and contextually relevant outputs, making it a key component in various applications, from translation to healthcare. This innovation has set a new benchmark in the field, demonstrating the transformative power of AI in mimicking human-like text generation capabilities.

But Illia wanted to do more, “Google has a lot of benefits, and it’s a great place to learn, but it moves really slowly. I wanted to move faster. I wanted to build a startup.”

In 2017, along with Alexander Skidanov, he co-founded NEAR.AI, a startup focus purely on AI.

"It’s purely AI. And as we were doing our AI thing we tried a lot of stuff, but nothing worked particularly well. We didn’t have hundreds of millions of dollars of hardware to train models on, like some companies right now. But what we did have was a crowdsourcing system.

It was a system that allowed us to find students, specifically computer science students around the world. From China, Russia, Ukraine, Poland. Countries where students are from humble means, and they do work for us for cents, pretty much. So we would need to pay them. And it was actually hard to send them money. It’s really hard to do through a traditional financial system. And that’s where the blockchain came in."

Therefore, the team ended up brainstorming they way solve the problem, for paying the students. They explored research on smart contracts, cryptocurrencies, and blockchain technologies in search of an appropriate platform for their applications. However, they couldn't find one that matched their needs, prompting them to create the NEAR Protocol to meet their specific requirements.

NEAR Protocol grew fast, from 9 people to one of the most prominent blockchain protocols in the space. It now has over 70 million accounts and 1.12 billion transactions. NEAR is a flourishing ecosystem of decentralized applications (dApps) and projects. Illia Polosukhin emphasised doing things differently. The journey to NEAR involved extensive research with 300 blockchain projects to understand their challenges. The team focused on creating a developer environment first, known as the Dev Node, allowing for immediate development and iteration. They experimented with five different consensuses and three different charting designs. They were trying different things, implementing them and seeing how they behaved.

NEAR is a layer-1 blockchain that uses a Proof-of-Stake (PoS) consensus mechanism. NEAR Protocol emerges as a beacon of innovation within the blockchain space, heralding a new era of scalability and usability. At its core, it is designed to address the fundamental challenges that have plagued many existing blockchain platforms, namely scalability and user experience. By prioritizing these aspects, NEAR aims to unlock the full potential of decentralized applications and foster widespread adoption across industries.

One key feature that sets NEAR Protocol apart is its sharding mechanism, Nightshade. Sharding is a scaling solution that partitions the blockchain network into smaller, more manageable segments called shards. This approach allows NEAR Protocol to process transactions in parallel, significantly increasing throughput and reducing latency. As a result, the NEAR Protocol can support a higher volume of transactions without compromising on decentralization or security.

Moreover, NEAR Protocol distinguishes itself through its developer and user-friendly infrastructure. Recognizing the importance of empowering developers to build on the blockchain, NEAR offers a suite of tools, libraries, and documentation to streamline the development process, making it easy to get started, even for those without a background in smart contracts or Rust programming. For end users, innovations like human-readable account names, simple sign-up methods, account abstraction and low transaction fees significantly improve the experience and help fostering wider adoption of blockchain technology.

At the heart of NEAR's success lies the expertise and leadership of individuals like Illia Polosukhin. With a background rooted in artificial intelligence, Illia brings a unique perspective and skill set to the table.

NEAR has just announced that it will hire AI engineers as part of its 3-6 month roadmap to building user-owned AI. The convergence of AI and blockchain is currently a significant point of discussion, with NEAR striving to be a pioneer in this realm. Illia Polosukhin attended a panel at Nivida’s conference titled ‘Transforming AI’ last month, alongside with the authors of "Attention Is All You Need," the groundbreaking paper that introduced the transformer neural network architecture as we mentioned above.

“NEAR has the most users in the world in the blockchain space. We’re talking about multiple millions of daily users who don’t even know they’re using blockchain. But they’re actually interfacing with programmable money and programmable value. And now we’re bringing back some of those AI tools to generate more data.

The way we’re rewarding creators right now is broken. And the only way to fix that is to leverage programmable money, programmable value, and blockchain. You need the basic primitive of programmable money because it’s what allows users and groups to coordinate people at scale.”

Illia envisions creating a version of the open web where programmable value supports a positive feedback loop, cultivating a new economy enriched by equitable participation. For instance, he mentioned that NEAR is investigating ways for users to contribute their own data to enhance AI models. Utilising blockchain to record IP, attribution, and royalties and more. These efforts illustrate how the future AI economy could benefit both creators and users by allowing greater participation and collaboration supported by Web3 infrastructure and incentives.

NEAR has plans to utilize NEAR.AI as a platform for revealing AI information and is set to make a public announcement in May. NEAR posits that while profit-driven companies will inevitably be driven by revenue incentives, its token economy will cultivate value within an open, unrestricted ecosystem.

Illia also mentioned in his speech at 2024 Hong Kong Web3 Festival that generative AI is becoming a tool for large-scale manipulation of people, and the economic situation of large companies leads to distortion of incentive mechanisms. After achieving the target market share, it is necessary to increase the average income of each user to obtain more user value. As a tool, Web3 can motivate people, create opportunities, and provide enough computing and data resources to build competitive models. If you want to integrate AI and Web3, you need to let a large number of AI tools play a role in the Web3 world. Since language models can directly interact with society, the creation and manipulation of false information become a major problem. However, AI is not the core of this problem, how to use cryptography and onchain reputation to solve this problem should be the focus of people's attention.

“I think AI right now will become a tool to break a lot of the current societal systems. So you can [distributed denial-of-service] the legal system with frivolous lawsuits. You can create fake candidates and try to get them elected. You can create fake companies and raise money for them.

I'm not even talking about the more sophisticated stuff. And so our plan is to organize a hackathon to get people to break all the stuff and then suggest solutions. And the reality is most of the solutions will use Web3 because you need authentication, you need identity, you need traceability, you need all those pieces embedded across the whole spectrum of our society.”

Learn more about NEAR: https://near.org/about

Join InFancy Community to Claim Your NEAR: https://t.me/InFancy_Community

We dedicated to AI-generated art and AI tools, InFancy.AI is committed to sharing and exploring models, prompts, and the latest developments in AI. Join us now!

Channel | Community | Twitter | Website