We’ve seen algorithms become the lifeblood of the internet - driving infinite recommendations on top of infinite feeds. In web2, these algorithms are complete black boxes (end-to-end), used for one specific application, and manipulated by some interested party. No one knows exactly what inputs are used, what scores are given to people and content, or what models are used for curation. And you can be sure there’s no API to run their models and get their outputs yourself.
Web3 algorithms are still nascent, but we’re close to a tipping point due to a combination of:
Modular Evolution: Web3 protocols are being optimized/simplified while actually emphasizing modular composability. That means anything that was the protocol team’s choice is now up to individual communities - everything from pricing curves, token and voting permissions, and customizable and conditional token marketplaces.
Maturing Data Infra: Everyone and their grandma knows how to do auto-generated decoded tables already. Data providers like Dune are maturing to compete at the aggregated tables level instead, across many chains.
All this means more data, more cross-app and cross-domain intricacies, and more powerful ways to index and analyze data. It’s inevitable that we’ll see this data leveraged to create curated and tiered experiences for communities and users alike.
This is the first of many Web3 data guides I plan on doing. The goal here is to explain the protocol from a data analyst/engineer’s perspective. These guides aim to bring transparency to the data you see on dashboards, as well as awareness of how well (or not well) protocols are built for data analysis.
In these guides, I’ll cover protocols in three sections:
All data is sourced from Dune Analytics, a platform for building cross-chain queries and dashboards - most of which is open source.
We’ve seen various combinations of NFT (ERC721) auctions, ERC20 airdrops, open sales of both, tiered access to the open sales, etc. - all under the guise of making you a “member of the community”. Some have worked well, while others have been flaming piles of garbage.
I believe each monetization method has its place, but we need to be more intentional about:
In this piece, I want to propose some frameworks and implementation tools/examples for monetizing membership across clusters in a community.
If you’re completely new to SQL and Ethereum I recommend starting with my full beginners’ overview or breaking down an Ethereum transaction first. Feel free to support my work by buying a Data Landscape edition at the end of the article, or sending a tip to ilemi.eth!
Most explanations of storage and state on Ethereum are diagram heavy and don’t get into the meat and bones of how to access or work with that data. I’m going to avoid the diagrams and theoretical explanations, but you should still have some understanding of data tries so skim this and this if you know absolutely nothing about how data gets into blocks.
Essentially, what you need to know is that every time you make a transaction with a contract involved, there will be state and storage changes saved to the block. State changes involve things like an addresses’ Ether balance or Nonce, storage changes involve storage slots on contracts such as your balance of ERC20 tokens in a contract.
To access these changes from an Ethereum node, you’ll need to use a replay VM - meaning you have to replay a transaction or block to get the raw diffs in state and storage. This is easily done using Infura or Alchemy, without running your own node. You can go to Alchemy’s API composer to play with the
trace_replayTransaction endpoint (here). If you want to read more about how traces works, read the standards here - but I’ll run through one example.
Welcome! I’m assuming you’re a data analyst who is new to web3, starting to build your web3 analytics team, or have just taken an interest in web3 data in general. Either way, you should already be loosely familiar with how APIs, databases, transformations, and models work in web2.
For this inaugural guide, I’m going to try to keep things very concise and highlight my thoughts across these three pillars:
This is probably a good time to say that this is all just my view of the space, and I’ve showcased only a few facets of the tools/communities mentioned below.
Hopefully, you're familiar with the Web3 product stack, and why it's considered composable. If not, then here's a quick refresher that explains the following layers:
Another precursor for this article that will be helpful is how web3 communities create tokens that become the membership structure that platforms are built off of, whereas web2 communities are forced into membership structures of existing platforms.
Having a token brings a community together loosely with a financial stake, but composable membership holds a community together long term with social capital. That's a lot of buzzwords, so let’s explore this in three parts:
Our data and actions give us identity and relationships, but having a social graph does not give you community. Put in a Web3 context, a group of people holding or trading the same token does not mean you are a community - at best, it gives you clusters. **An actual community has fairly clear boundaries for growth (in more than a financial sense). You've probably seen some variation on the chart below...
It's been almost one year since I really entered crypto, which I count as the day I first subscribed to @bankless. It's been quite a fun journey, and I'd love to share the path I took and some of the lessons learned.
Before I entered crypto, I worked at a large investment bank in their rotational grad program. Over two years, I worked with different teams on everything from managing the bank's liquidity to our commercial real estate portfolio.
User experience (UX) describes how people feel when they interact with a system or service and encompasses several factors including usability, design, marketing, accessibility, performance, comfort, and utility. Don Norman once said,
“Everything has a personality; everything sends an emotional signal. Even where this was not the intention of the designer, the people who view the website infer personalities and experience emotions. Bad websites have horrible personalities and instill horrid emotional states in their users, usually unwittingly. We need to design things — products, websites, services — to convey whatever personality and emotions are desired.”
Ethereum's personality is someone who is extremely inscrutable and easily misunderstood. To make matters worse, most users don't even think about it as interacting with Ethereum when they're using your interface or a wallet. If you're ever in the live chat of an Artblock's auction, you'll notice as soon as an auction ends there are at least a dozen people complaining that it is Metamask's fault that they didn't get a mint. I think in the last year the UX of many dapps on Ethereum has improved greatly in both product interaction and explainability of transactions. For the most part, dapps don't just leave you with a loading spinner after you sign the transaction anymore.
Even with the design of dapps is generally improving, I'm not sure how deep UX research has gone yet. When I see data analytics or research on various protocols, users are mostly treated as homogenous. This is somewhat shifting with some of the analysis I've seen of Uniswap V3 liquidity providers and Rabbithole Questers, but even those are still heavily focused on just confirmed transactions on-chain. From my own experience, most of the emotion evoked and behavioral quirks happen when I'm submitting, waiting, speeding up, or canceling my transaction. For some applications, the user might leave and go do something else after submitting a transaction. But for a product like Artblock's auctions, they're going to stay around until the confirmation has happened, likely checking anything they can for updates and with compounding anxiety.
Airdrops have come a long way in the last year, most notably starting from Uniswap's 400 UNI token airdrop in September of 2020. From there, we've grown into airdrops that are calculated based on the frequency and volume of interactions with different facets of a protocol. Below is one such example:
We've also seen NFT drops recently that rely on more standardized tools built for quickly setting rules/requirements and rewards/distribution. None of these methodologies are bad, but I do believe there's still a lot of room for improvement. This could come from the distribution model, drop frequency, or even the token rules themselves. In this proposal, I'll be focusing on a new distribution model, one that combines data from user interactions in the Mirror $WRITE race, on Twitter, and in Mirror-related Ethereum transactions.
In my previous post on digital identity, I mentioned that "the tokenization of these graph shards could take many forms and will likely be layered upon by proof tokens." I believe that sharded graph identity approach requires coming up with a community-specific reputation score that measures how influential a certain person has been in expanding a specific network. While some reputation scores may be more set in terms of having to have done X or Y actions, this score captures a users reputation in the context of other users in a more fluid manner - acting like a signal rather than a badge.
Typically in Web2, users are "rewarded" by an algorithm which will highlight them based on the engagement and attention they bring to the platform. Thus, their reputation score is just the number of likes or followers they have - regardless of who those vanity metrics come from. In Web3, we typically reward with tokens representing the value of the protocol or product. These tokens also carry a lot of power in the form of voting and other privileges. As such, a score that signals not only influence but also how supportive or aligned a user is with the rest of the community will become increasingly important over time.
For this post, we will be focusing on creating a reputation score for Mirror users (voters, writers, contributors) based on where each user sits on an interaction-based social graph across Ethereum, Twitter, and Mirror data. I chose Mirror for three reasons:
A decade or two ago, your digital identity didn't mean very much. It was represented more by your email address than any social media account you had. Fast forward to today, and we've seen an increased focus on how we appear in online media by everyone: yourself, your friends, and your employers. Most of our day-to-day interactions exist in digital accounts on someone's database.
Yet all-in-all, your digital worth is measured by the engagement (and advertising revenue) garnered. Sometimes you get a share of that revenue, but for 99% of us, the value of our digital identity is locked within the platform. This doesn't just apply to Facebook, Twitter, or Reddit - the "platform" can be our workplace as well.
Let's say you've worked at a company for a few years, and you've decided to move on. When you leave the company, your reputation still stands with individuals you've worked with and in the many programs/emails you've created, but otherwise, the company's reputation is your reputation. This dynamic has slowly changed with the rise of the internet - hence the rise of the personal brand. However, your personal brand is still built off of your own word (which may or may not have much value when first starting out).
Our digital identity exists in fragments, it isn't flexible past the platform it was built on, and it has little reusability other than cross-platform authentication. We don't own our digital identity, and it's not composable at all. Now, what if you could own all of the pieces of your digital identity permanently, while also controlling who you reveal that data to and how it's represented? This would enable us to tell more compelling stories and have a user-controlled value function for identity. I think we're (finally) not so far off from a future where all of that is possible.
Ever since sharding and the internet of blockchains ideas were first introduced, maintaining composability has been a hot topic of discussion. Let's try to better understand the role bridges play in keeping our ecosystem composable.
While I've written about composability of dapps and wallets before, I don't believe I've explicitly defined the word "composable." Generally, I think composability is comprised of existence, flexibility, and reusability. As a pre-condition, everything should be accessible in a permissionless manner (i.e. sufficiently decentralized).
I believe that right now, most people think of bridges as "how can I get token A transferred from blockchain X to blockchain Y," or slightly fancier as "how can I swap token A on blockchain X for token B on blockchain Y."