Stanford professor Dan Boneh teaches “Blockchain Primitives: Cryptography and Consensus,” providing an introduction to the cryptographic foundation of blockchains and how developers can use them to build new types of applications.
Andreessen Horowitz’s Crypto Startup School brought together 45 participants from around the U.S. and overseas in a seven-week course to learn how to build crypto companies. Andreessen Horowitz is partnering with TechCrunch to release the online version of the course over the next few weeks.
Find more Crypto Startup School videos plus additional reading and info:
[Music] So welcome everybody I’m really excited That this school is happening this is an Amazing opportunity I’m actually really Excited to see what kind of apps you Guys build and I’m looking forward to This week seven where we get to see Everything you’ve you’ve designed so What I wanted to do today in this and This Cup in the next hour basically is Tell you a little bit about the Technology behind blockchains I’m really Excited about this technology and that Technically it’s just from a technical Point of view it’s fascinating to work In the basic the basic issue is a Building decentralized applications is Just technically much harder than Building centralized applications which Makes it very interesting for someone Like me So I teach this stuff you know I kind of Live this on a daily basis this is gonna Be a pretty high level talk this is a Rabbit hole that goes pretty deep so I’m Happy to go kind of to as much depth as You’d like maybe after after the Presentation I should also say that We’ve been teaching this this class at Stanford now for what is it five or six Years and pretty much the tenants in the Class kind of follows the Bitcoin price It’s pretty pretty pretty normal yeah a Bad day students don’t show up on a good
Day they do show up all right so let’s Get started and I want to get started Just kind of by explaining my view of What a what a blockchain is so I kind of I kind of like to think of it in terms Of four layers yeah so the bottom layer We have the consensus layer which just Makes sure that every everybody sees the Same thing on top of the consensus Layers this is layer layer one on top of The consensus layer we have what I would Call layer 1 point 5 which is the Compute layer and here I stole Chris’s Term the blockchain computer I love this Term so thanks for inventing that Chris So it’s called a blockchain computer This is kind of the operating system of The blockchain On which you write applications and then Layer two is basically where the Applications run and typically Applications will be written in Languages like solidity or move movies Libera Libre language or Motoko and so On and then on top of the applications These are applications that actually Run on the blockchain computer and on Top of those you have sort of user Facing systems that kind of run in the Cloud and talk to the blockchain and Talk to to end users yeah so those are Kind of the four layers I have to say The consensus layer is getting pretty Pretty well understood that’s kind of
There’s still a lot of work here but This is kind of not where the most Exciting action is most of the exciting Action actually is in building Applications in fact these layer 2 Applications are now starting to be Called layer 2 layer 2 block chains and So that’s kind of where most of the Action is and I imagine that’s most of What’s where most of you are going to be Working in now when I showed this slide At some point somebody asks me these What you blockchain people you don’t Know how to count and the the reason It’s layer 1 and layer 1.5 is because Often these two are kind of combined Together but I cannot like to separate Them because really there’s a consensus Issue and there’s a computing issue and Literally we could talk about each one Of those layers for weeks and so but we Don’t have weeks so I’m just gonna say a Few words about each layer and then We’ll move on to more programmatic Advice ok so let’s start with consensus So first of all the consensus layer Again makes sure everybody sees the same Data so it provides persistence so once Data gets added to the blockchain it can Never be removed and there’s an asterisk There because there’s a caveat the Caveat is a 51% attack so under certain Assumptions about malicious behavior the Data can never be removed consensus
Means that basically all the Participants see all the data the reason There’s two stars there is because when We talk about all the participants Seeing the same data we mean that that Is true perhaps other than the last few Blocks that were added So I mean you may have to wait a few Blocks for every month to see everything But otherwise everybody sees the same Data live this means everybody can add Data to the blockchain so it doesn’t get Stuck and finally there’s this notion of Openness which means that anyone can be A participant anyone can write to the Blockchain and there’s no authentication That’s needed to become a member and and Right to the blockchain the reason There’s a question mark there is because Some blockchain is provide openness and Some blockchains Some block T’s are open block chains and Some are are permissions where not Everyone can write so that’s the Consensus layer I didn’t want to say a little bit about Just a few words about what what what it Does and so this problem consensus is Not a new problem this is an ancient Ancient problem and by ancient I mean From the 1980s well you know you have Right you have systems that people run Multiple servers and their data centers And these data sent these systems have
To see the same data right Bank of America have has a huge number of Machines they all have to make sure they See the same amount of funds in my Account and so that’s this problem is Called what’s called state machine Replication Trane’s traditionally it’s Actually not called a blockchain it’s Called state machine replication the Difference is that in the traditional World there’s a known number of servers And all these servers have been pre Authorized so they’ve all been given Sort of cryptographic material that Allows them to authenticate to one Another the difference with the Blockchain world or the open consensus World is as we said actually there’s an Unknown number of participants who can Write to the blockchain and they’re not Authenticated anyone can participate you Don’t have to do anything in order to Participate and the surprising thing is That in fact people have asked can we do Consensus in these settings and there’s An impossibility proof that says you Cannot achieve consensus if you don’t Have authentication or you don’t have Any you don’t know how many participants Are in the network and sort of Nakamoto’s innovation is a way to get Around to get around this impossibility Result and I’ll talk about that in just A second okay so now that we understand
What what the innovation is let’s just Talk for just briefly about how blocks Get added to the blockchain I’m sure Many of you have seen this before But it’s still kind of fun to go through So we have our cute miners over there And we have our participants each Participant has a secret key which goes Back to what Chris was saying this Secret key signifies ownership right so You own your tokens you own your data And no one can take it away from you Unless you sign it away with using your Secret key or someone compels you to Give to give away your secret key so how Does this work well so these Participants these users basically sign Transactions to use their secret keys to Sign transactions this cent they send These transactions to the miners now the Consensus protocol basically chooses a Random leader among these these Participants the random leader orders The transactions into a block and then The block Gets posted on to the blockchain and Importantly the leader gets paid by a Native currency to the blockchain right So in this case they get a p2 eath and Then the other ver the other miners Actually verify that the block is valid If it’s not valid the block sort of gets Discarded there’s one thing that I Wanted to point out here which is kind
Of important which is you realize the Miner this leader that got elected the Leader is the one who decided the order Of the transactions in a block just Think about that for a minute The leader the miner that built the Block decided on the order of Transactions that is an extremely Powerful position to be in because if a Certain piece of bad news comes out About the blockchain you could imagine People are lining up to sell their Currencies and who gets to sell first is Determined by the miner this is a this Is a power that can be sold right which Is not a good property we’d like to have So ideally we’d like to have a Blockchain where in fact the order is Not determined by the miner but budget But determined by some randomness but This is not currently supported so if You’re looking for interesting questions To think about this is a really a Fascinating one which is how do we make Sure that no one not even no single Miner has the power to order Transactions as they get put on the Blockchain yet this is kind of an Important point okay so we have this This mechanism and then you know a new Set of transactions is generated a new Leader is elected and the block and a New block is created now you realize and Then this new leader by the way also
Gets paid to eath now you see why the Impossibility results came about because If there is no authentication and no Nobody knows how many participants there Are what I’m gonna do is let’s say I’m The blue miner over there what I’m gonna Do is I’m gonna pretend like I’m a Thousand different miners even though I’m a single entity I’m gonna pretend Like I’m a thousand different miners and Then these other two miners will hardly Ever get chosen I’m gonna be elected Every time right there’ll be a thousand And two miners overall I get elected Every time so I always get to collect The two eath the block rewards for Creating these blocks so that attack is Called what that’s got thank you that’s All together now So that’s called the simple attack and The whole point the impossibility Resolved by the way said if you don’t Have authentication consensus is Impossible because you’re going to be Vulnerable to a Sybil attack and the Beauty of nakamoto’s innovation is that We can prevent Sybil attacks by forcing Some sort of commitments to constraint To resources right so we’re going to Force the miners to commit the resources And you guys know the story that in in Bitcoin the resource we’re committing to Is proof of work right so if I want to Pretend to be a thousand miners I have
To buy a thousand machines and that’s Expensive yeah of course proof of work Has this is basically what’s been used In the first generation blockchains I Think we’re gonna be moving away from Proof of work it’s proven to be Problematic because of its it’s Relatively slow it burns a lot of energy So all the criticism that you that we Read about in the press usually Criticizes how much energy these things Are wasting and the answer is well no That was just a first attempt at Building a blockchain we’re moving away From proof of work and actually all the Other ideas are much faster and they Don’t burn energy or hardly burn any Energy yeah so first thing is when Someone complains about energy you tell Them that well that was the first Experiments the future is not is not Gonna be much more efficient energy wise So we have a lot of block J’s that are Proof of stake based systems so again They prevent Sibyl attacks by forcing People to commit to funds rather than The commit to CPUs proof of space is a Beautiful beautiful idea where you say Instead of committing to computing power You have to commit to disks yeah and the Nice thing about committing to disk so Disk space the next thing about Communicate this to disk space is that That doesn’t take much energy right
Storing data doesn’t take much energy You just have to show that you’re Storing certain blocks when when needed And then there are some other beautiful Ideas due to Avalanche and others so There’s lots of explorations of the of The consensus space just so you’re aware But that’s actually again not quite Where the action is as I said most of The action is in layer two in building Applications on top of block genes okay So let’s talk about that so but the Blockchain computer so here I’ll go Quickly through this so basically you Write applications you post them on the Blockchain and then these applications Basically respond to events from Outside these events are called Transactions right so people send Transactions to these programs that are Running on the blockchain and that and The applications respond yeah so the Beauty of running applications on a Blockchain is a essentially all the code Is open source and I have to tell you This is so refreshing working in the Space where I talked to a lot of Blockchain projects and nobody ever Tells me you know we can’t tell you what We’re doing it’s a trade secret which is What I usually hear everybody’s very Open about what they do all the code is Open source all the ideas are out there Everybody publishes as fast as they can
So it’s just so refreshing it’s so much Fun so much fun to work in the space There are really no secrets because Everything is based on trust right I’m Not gonna trust you until you unless you Tell me how the system actually works So everybody’s incentivized to share Their ideas and share their code so Everything is open-source everything is Verifiable so we have this property I Like to call that transparency where the World can see exactly how things behave And then we have also public Verifiability where I can look on the Blockchain and see exactly that all the Rules are being followed correctly that The code is actually being executed the Way it’s supposed to execute and no one Is deviating from the rules right so Again in the real world you think about That we don’t have public verifiability Where most of the time we have to trust That banks do what they’re supposed to Do or the Fed does what it’s supposed to Do most of the time we have it’s based On trust whereas here you could just Look and verify for yourself that things Are progressing correctly so let’s go Through quickly how you deploy Applications on the blockchain so we’ll Do that very quickly so you write your Code you create an application you write That in solidity you compile it into the EVM for example if you’re using theorem
Then you post your code onto the Blockchain with what so the this orange Thing is the code this blue thing is Kind of the initial state for the Application and then somebody sends a Transaction to the application and Basically that causes a state change in The application so maybe some funds Moved or maybe Chris Nixon got to own CDX and Comm right so that that that’s The type of transaction that is Processed and then you know another Event happens and another another state Change happens and so on and so forth Now in preparation for what’s coming I have to tell you that these as I think Chris said that the theorem network and Transit can process something like seven Transactions a second That’s way too slow for running all the World’s application so really all the Focus today is on scalability and Basically touching the blockchain as Little as possible yeah and I’m gonna Talk about techniques for doing that in Just a minute but essentially when you Build your layer two apps part of the Goal is to avoid touching the blockchain As much as possible so that things Actually scale or I should say to be Precise to avoid touching the layer one Blockchain so the layer two block chains Are much more performant okay good right So there are different execution
Environments these blockchain computers Provide different interfaces so for Example Bitcoin provides bitcoin Script Which is a very restricted language so In particular you don’t have loops if You don’t have loops you can tell Exactly how long every instructor every Program is gonna take to run so that has Benefits but they’re the same so it’s Good enough for certain things but at The same time there are lots of things You’d like to code up that you simply Cannot code in in Bitcoin it’s Interesting like in our class on Payments and lecture on payment channels I can tell you we devote about 45 Minutes of the lecture to explaining how Pay me pay me channels work on Bitcoin It’s fairly complicated and then we Devote about two minutes to explain how It works on aetherium because it’s Trivial yeah once you have a general Computing environment building payment Channels are called state channels are Much much much simpler simpler to build So yeah so it’s restricted but it’s Amazing and how much it can still do the Theorem is a general-purpose computing Environment like we said you program it In solidity one thing that’s that’s Really important to remember is that any Data you put on the blockchain is really Expensive yeah you have to pay to put Money on the blockchain and it’s kind of
Remarkable if you’re gonna build Blockchain application on a theorem you Will not believe how much time you’re Gonna spend removing every single bytes From the from from the blockchain from Your app yeah so you’re gonna do all These tricks crazy programming tricks Just to reduce the amount of storage on A blockchain today when you want to Write They data to the blockchain you have to Pay when you write and that’s it in the Future you’re actually gonna have to pay Continuously for storing data on the Block Yes so your goal would be to either Expire data as quickly as possible or Avoid writing it if even in the first Place I can I guarantee you as you’re Going to be developing you’re gonna be Spending most of your time in trying to Remove data and not writing it to the Chain itself so far so good Okay excellent all right so yes there Are lots of execution environments one Thing that I wanted to point out is Ethier aim uses the EVM as a virtual Machine which is kind of an interesting Choice for a virtual machine you can see That many of the new modern many of them Modern blockchain is actually I’m moving To web assembly so it’s a particular Byte code format and the beauty of web Assembly is you get to benefit from all
The software development tools that were Developed for the web where you can Write your application in your favorite Language compile it to web assembly and It will just work on the on the Blockchain yeah so kind of modern Chi Modern block chains are gonna have much Much more much a much richer development Environment to work with just keep that To keep that in mind okay good right so The application layer is kind of the Most exciting part as I said there are Lots and lots of applications available Already today it’s actually really fun To write these applications I have to Tell you it’s really really fun the one Thing you have to be careful with is if You make a mistake if you make a bug in Your application you can easily end up Locking up 50 million dollars That no one can get ahold of yeah so Mistakes here are really really costly So the programming language you choose Is kind of important there’s a story I Like to tell about about solidity so Every every time you know so you know so Literally it’s a wonderful first example Of a blockchain programming language but I can tell you every year in our final Exam we post an example contract where We asked us we deliberately put five Bugs in the program and we asked the Students to find the bugs that’s our Final exam question and I would like to
Say that I know how to program it slowly But nevertheless every year the students Find 10 bugs in the code yeah and so That’s just an example of yes as an Example of what it takes to write secure Secure code on on these boxes so Interestingly more modern languages like Move and Motoko they’re kind of trying To address So move is specifically designed to be Easy to verify yeah so there’s a lot of Work that goes into building automated Verification tools so hopefully this Problem this problem will be solved in The future okay good so I think maybe I Just wanted to mention the last thing is Effectively the layer four application At the user facing services what they do Is again we have our consensus layer Here are blocked in computer and we have Our apps running this is running on the Blockchain but it’s important to Remember that really what the user is Talk the users don’t talk to the Blockchain what the users talk to our Application servers that run in the Cloud yeah and so they just you know so The web server is actually one year in The cloud and they just get their data From the blockchain so the blockchain is In some sense the ground truth but Really everything is replicated into Databases that live in the cloud and End-users just see just see that yes
Again if you go to a handshake and you Try to to reserve to buy a domain name It first gets recorded here and then Gets pushed onto the blockchain okay but So you you probably all know there’s a Thriving ecosystem of applications Running on the blockchain just picked Every time I put up this picture I’m Just kind of amazed at how active the Space is so and this is by the way only This decentralized finance world it’s Not even there applications beyond defy This is just defy so I think Chris Already pointed like one of my favorites Defy applications is maker Dowell it’s Just an amazing idea like how to build a Stable currency using collateral that Lives on the blockchain it’s just like When you read it you just become happy Because it’s such a beautiful idea that It’s it’s a lot of fun just kind of Study and analyze how it works so yeah I Encourage you to read about maker Dow It’s just a really really beautiful idea And it’s so far seems to be seems to be Working seems to be working quite well So let’s make heard out Effectively what they did is they took The US Fed policy the US Federal Reserve And just implemented in an algorithm as A as a short description okay and then You can talk about calm lending Platforms like compound those are very Interesting too and these applications
Literally you can lock you can go to the Blockchain and just read The code like you know I stopped Actually reading white papers to Describe how the code works I just Literally download the code read it and See exactly how a compound works it’s Not that hard it’s not that much code And you can just understand it best by Just looking at the code so again I Encourage you to try not to read the White papers or maybe read them just his Background but then just go to ether Scan and get download the code and just Read it it’s very clear usually it’s Very nicely commented and you can see Exactly how things operate very Instructive okay so yeah so we’re gonna Talk about applications actually next Week we’re gonna talk about lots of Applications in the coming weeks out Even and so I’m not gonna say more about That here what I wanted to do now is do A digression how are we how we doing That time okay fantastic All right keep going and I wanted to Switch switch gears and talk about Cryptographic primitives so I’m a Cryptographer so for me this is you know Have to say these block chains are like A godsend because crypto is like so much Fun now Right everything that we do gets Implemented and deployed and there’s
Some things that we’ve deployed that are Now protecting billions of dollars which Is exciting and frightening at the same Time so what I wanted to do is tell you A little bit about cryptographic Primitives that are useful on the Blockchain So I hope these are things that you know You will put to use in your own projects And so again this is a pretty high-level Description I’m really happy to go to as Much depth as you’d like on these things But which I’m gonna try to get through This in like ten minutes So these are big topics each one of These circles literally is a huge topic But we’re gonna do it quickly and if You’d like to talk more I’m happy to Stick around and talk more about that Afterwards yeah so I wanted to tell you A little bit about digital signatures And in particularly aggregating digital Signatures then we’ll talk about merkel Commitments and finally we’ll use those Two things to build what are called Succinct zero knowledge proof systems They’re also known as snarks it’s a Great name okay so let’s start let’s Start with digital signatures so I Imagine actually many of you already Familiar with digital signatures so let Me go through this relatively quickly so In the physical world you know what it What a signature is you just sign your
Chuck but you think about that for a Minute and you realize wait a minute Can’t possibly work in the digital world Because in the digital world I could Just copy the bits I can just copy the Bits into another check and that would Not be cool Yeah so clearly what we do in the Physical world simply cannot work in a Digital world so we have to build Signatures slightly differently and so This actually was a people were stuck in The when these ideas were developed back In the 80s it wasn’t clear how to sign In the digital world because of this Problem and so the idea it’s actually do To Michael Rabin is to basically make The signature itself be a function of The data being signed yeah so it’s not Like the physical world signature we’re Here were in the digital world you Actually look at what you’re assigning That becomes an input to the signing Algorithm along with the secret signing Key and and then these two things Together the signing key and the data Being signed they’re the things that Produce the signature okay so now the Signature depends on the data then given A signature you can actually feed it Into a verification algorithm the Verification algorithm takes what’s Called a public key okay so that’s a Matching pair to the private key so the
Signature the data and the public key go Into the verification algorithm and the Ref occasionally where them says yes Valid or invalid okay so that’s how Digital signature works digital Signatures work the thing that’s Important to understand is if you don’t Have the signing key or if somebody Doesn’t have your signing key they Cannot sign in your behalf period yeah It doesn’t matter how many signatures They see if they don’t have your signing Key they simply cannot generate Signatures on your behalf so that’s what A digital signature is signatures are Used everywhere on the blockchain yes so Do use to authorise transactions They’re used in governance votes I guess There was a maker down governance votes Right so the way that worked is all the Folks participating and their evidence Use their signing key to sign to sign The proposal that was being voted on and It’s also there it’s also used in Consensus protocol and particularly in Proof of stake consensus protocols all The verifiers basically sign and if Enough signs then the block becomes real So yeah so signatures are kind of used Everywhere you can see that when these Transactions get their transactions are Signed they get posted on the block And then all the verifiers have to Verify all the signatures if you think
About it for a minute you realize that’s A little crazy right if once minor Verifies a signature we know all the Signal we know that it might the minor Says the signatures are valid why is it Oh that all the other miners now have to Repeat the work and verify the Signatures for themselves but this is How this this is how it works today they Keep that in mind because we’re going to Come back to that in a minute Okay so that says digital signature I Wanted to also mention this space-saving Idea so there’s something called Signature aggregation where you can take For certain signatures you can take a Whole bunch of messages Signed by different people and Effectively compress all the signatures Into a single signature this is called An aggregate signature that’s a purely Compression mechanism you think a whole Bunch of signatures compress them into One and now when you want to verify you Need to know all their public keys you Need to know the transaction data and You only need this one aggregate Signature and that allows you to verify That all the messages were properly Signed okay so the point of this is that You don’t need to store all the Signatures on the blockchain you can Just compress all them and just store The aggregate okay so again it’s a very
Useful compression mechanism so I just Wanted to mention there’s a blockchain Called Shia that you can see when you Post transactions on shia what the miner Does is he takes everybody’s individuals Signatures compresses them into the Aggregate and only stores the data in a Single aggregate okay so this is a Really nice compression mechanism to Reduce the amount of data stored on the Blockchain and remember our goal is to Reduce the amount of data as much as Possible okay good so keep that in mind There is this nice compression Compression trick that everybody can Take advantage of all right the next Thing I want to talk about is Merkel Commitments actually I am really curious How many of you know how Merkel Commitments work ah okay not too many so Okay so I’ll explain it at a high level So let me explain what a cryptographic Commitment is yeah it’s uh yeah so What’s a cryptographic commitment so a Cryptographic commitments basically Functions like an envelope all right so Imagine imagine we wanted to run a Sealed bid auction how does a sealed bid Auction work in the real world right Everybody takes their bids Sticks them into an envelope seals the Envelope sends the envelope to the Auctioneer The auctioneer collects all the
Envelopes right once they’re all at the Auctioneers hands he opens up all the Envelopes and then decides who won the Allahu anhu won the auction yes so well A cryptographic commitment does exactly That right so you can allow you can put Data inside of an envelope send it to The auctioneer and then later on the Auctioneer can open the commitment and Recover the data all right that’s Basically what a cryptographic Commitment allows you to do very very Important primitive in the blockchain World so let’s define it more precisely Basically there’s an algorithm that will Take some data and will produce a Commitment string that’s the thing That’s gonna be sent to the auctioneer And then there’s some opening string That’s gonna be kept secret then there’s A verification algorithm that’s gonna be Run by the auction house that allows you To basically say oh the data that was in The commitment is this data over here And the opening string is the proof that The data is correct okay so I can commit And then later on I can open but I can Only open and the commitment in one Particular way so the properties of a Commitment schema is exactly that it’s There’s a binding property that says I Cannot I cannot open the commitment in Two different ways once I’ve committed To something I’m bound to that data and
Then the hiding property says that if I Give you the commitment string you have No idea what data I committed to yeah It’s in the envelope and you cannot see It the reason this is such an important Primitive is what I can do is I can Commit to data on the blockchain I can Literally put data on the blockchain I’m Bad I am bound to that data I can no Longer change what it is because of the Binding property but because the Commitment is hiding nobody knows what Data I post it onto the blockchain yeah So that’s that that’s why this is such a Useful mechanism in this context okay so There’s a very famous commitment scheme Well actually there’s a Merkel tree Commitment which as I described it it’s Not hiding but it could easily be made Hiding where what you can do is you can Take a whole bunch of values and produce A very short commitment to those values Very sure when I say very short I mean Like 32 bytes just 32 bytes there could Be a million elements and the leaves Here yeah so this could be gigabytes of Data And yet you’re compressing it to 32 Bytes in a way that you’re committed to All of these values and more importantly Later on if I want to convince you that Element number 4 is x4 I can produce a Very very short proof it’s like a Kilobyte proof that will convince you
That x4 is really what what I committed To yeah so I have a huge set of data I Commit to this huge set of data but now I can prove to you I can open individual Cells and prove to you very efficiently That the cells were are correctly that What I showed you is the correct value In the cell okay so this is called a Merkel tree very important again very Important idea this is basically Deployed in all the block chains out There yeah So I wanted you to see how it works the One way to use it for example is just Convince someone quickly that payment Was made so imagine we have a thousand Transactions in one particular block ok So here’s one block we have a thousand Transactions we actually don’t write all The transactions to the blockchain Instead we’re just gonna write a Merkel Commitment to the blockchain so again we Minimize the amount of data on the Blockchain yes so we just write this Short hash so the blockchain but now if Alice wants to prove to Bob that she Really paid him to eath it’s really easy For her to do because she can produce This very short proof that says yes I This transaction is part of what was Committed to the block two to two on the Blockchain yeah so again Alice can Produce a very short proof to Bob Bob Can check the proof Rob has all the
Blockheads he can check the proof and be Convinced that Alice really paid him but You see again we’re moving data off of The chain which is kind of what the game All this game is about reduce the amount Of data on the chain itself ok good so That’s one application yeah easily do Giving proofs of payments the other Application I wanted to mention is that You can actually keep a whole database Off of the chain using this trick so Imagine we have a bank and the bank Actually maintains everybody’s balances Right so Alice has ten coins You know Bob has five coins and so on And so forth we can commit to this Entire database using a tiny commitment String 32 bytes that’s that’s the only Thing that’s gonna live on the Blockchain but now if Alice wants to Prove that her balance is 10 is actually Quite easy for her to give this proof That just by looking at this hash value Anyone can be convinced that she really Has ten ten tokens in her account yeah So again it’s a way to keep an entire Database off the chain that the first Thing that would come to mind when you Try to design a system like this is to Write the entire database onto the Blockchain what I’m telling you is you Should never ever do that Yeah that’s kind of not a good design It’s better to keep data off the
Blockchain there are some techniques Lots of technical issues come up in in In making this work there’s something Called the data availability problem That you have to get around so there’s Lots of technical issues in making this Work but this is a wonderful way to keep Data off of a blockchain okay now if a Transaction actually happens say Alice Paid Bob to tokens you can see Alice’s Balance went down and Bob’s balance went Up here I can show you again yeah you Can see Alice’s balance went up went Down Bob’s went up and then all we do Now we have to update the database so we Just commit again on the next block we Commit to the updated version of the Database yeah so every time there’s a Change we just commit to a new version And still everybody can prove that their Balances are what they’re supposed to do Supposed to be in the only thing that The sort of the program on the Blockchain needs to check is that is That this transition is a valid Transition all right so that’s how Merkel commitments are used again we’re Building towards towards a tool that’s That I’m gonna mention in a minute so This is an important component of that Um great so the last topic the last Primitive I want to talk about is what’s Called a zero knowledge proof system so These are these are kind of things that
Zero knowledge proof systems are the Sort of things that you wouldn’t believe Are possible if I just told you what They are you would go no no sir surely You can’t do that So maybe the easiest way for me to Explain what a zero knowledge proof System is I’ll explain it like I had to Give a talk to kindergarteners one time About a crypto So and I decided decided yeah well I Decided to explain zero knowledge proof Systems to them so so the question is How to do that and the example I came up With was was Where’s Waldo you guys know The Where’s Waldo puzzle right I’ll give You a picture with a thousand faces And I want to the challenges find Waldo So I want to prove to you in zero Knowledge that I know where Waldo is in This in this image yeah so I walk into a Room you showed me the where as well the Puzzle and I can see Waldo and I want to Prove to you and zero knowledge that I Know where where Waldo is yeah so that’s A puzzle for you you guys can think About how you do that but maybe I’ll Just tell you yeah so here’s one here’s One one simple protocol for doing this It requires a scissors yeah so what I do Is you give me the picture with a Thousand faces and I find Waldo and I Want to prove in zero knowledge that I Know where Waldo is can somebody see how
Would you how would you use scissors to Do it oh ooh but then they could Separate they can separate the two pages And then and then they’ll know where Waldo is I wouldn’t be zero knowledge But you’re on the right track what you Would do is you would take the scissors And simply cut Waldo out of the picture Out of the challenge Picture with a thousand faces and then You actually would turn around right and Then cut it behind your back and then You would rip the rest remaining Pictures so the that nothing is left of The right shredded nothing nothing is Left of the original picture and then You can give the Challenger you can give Them the picture of Waldo and Unfortunately they have to they have to Search your pockets before you do this To make sure you didn’t walk in with a Picture of Waldo but if if you’re ok With that then that’s kind of a zero Knowledge proof that that you know where Waldo is in he learns nothing he or she There nothing at all Learn nothing at all about where Waldo Is okay so the kindergarten is by the Way asked me well why do you want to do That in the first place so so that Didn’t work that didn’t work very well Better now it gives you an example of What does zero knowledge what is it yeah That didn’t work at all by the way that
You were not they weren’t they were not Impressed that but but but the point is I can prove that the amazing theorem Says that I can prove anything that can Be proved can also be proved in zero Knowledge yes if a certain fact has a Short proof I can convince you of that Fact without telling you anything about What the proof is yeah let me give you An example of that so let’s suppose so Here we have a proven and a verify right The prover wants to convince the Verifier of a certain fact so Let’s imagine this what the prover is Trying to prove is that he has a Signature on a particular statement so Maybe there’s a transaction in question This is called the statement that I’m Trying to prove I have a signature on This transaction and then there’s a Secret witness which is the actual Signature yeah what I’d like to do is Prove to the verifier that I know a Signature on this transaction in zero Knowledge which means that I learn I Leak nothing about the actual signature Okay so the statement is yes the verify Says the prover has signature on a Transaction the prover will send a proof To the verifier the verifier will either Accept or reject the proof if the Verifier accepts the proof he’s actually Convinced that the prover has a Signature but he learns nothing at all
About what the signature is yeah it’s an Amazing fact that this can be done but It actually can’t be done this is Something that was proved back in the 1980s but in a very theoretical fashion In that normally this would be quite Inefficient I have to tell you because Of the blockchain world in the last five To six years there’s been just Mind-boggling progress in making this Work and now these things are actually Actually implemented deployed there are There’s a whole software tool chain that Allows you to build to build these your Knowledge proof proof systems there’s a System called circum that where this is What we use in our courses where you can Actually program these statements and Then build easier and all this proof System systems automatically so it’s Really become a very applied and very Practical field so it’s amazing amazing To see this happen okay So what’s a statement as I said the Statement is just basically a program That’s applied two to two inputs the Statement and the witness and you can Prove that given a statement you have a Witness that makes the program output One yeah that’s kind of what the zero Knowledge proof system does great so What do we use oh yeah actually first of All so what are the properties of these Proof systems just so you understand
What they provide so first of all if the Statement is true the proof exists yeah There’s a way to convince the verify That’s completeness the proof has to be Short okay so it’s very important to Proof be short because these proofs are Going to go on the blockchain so we want Them to be short because we always try To save bytes on a blotchy okay so you Have to be short they have to be fast to Verify why do they have to be fast to Verify because all the miners have to Verify them so they have to be really Really cheap to verify and then they Also have to be relatively easy for the Prover to generate okay so if you have All these properties this is what’s Called a snark yeah so snark stands for Succinct non-interactive arguments of Knowledge yeah Forget that just remember it’s called a Snark it’s a really really cool name now It has to have some security properties In particular you shouldn’t be able to Prove a statement that’s false yeah That’s called soundness and then Optionally you could also have a zero Knowledge property which means that you Learn nothing about the witness if you Have the zero knowledge property this is Called a ZK stork okay so there are two Types there’s a snark and as a Zeke a Snark and they have different Applications so let me show you up in
That one application for a snark and let Me show you one application for a Zeke a Snark and then I’ll stop does that work Okay let’s do it all right so the Application for for a snark is actually Kind of a big deal now it’s called roll Up so roll up is now taken out this is An idea that’s taken over the world Yeah so these are called levels it’s now Even called a level two blockchain and So let me explain the idea behind Roll-up So I already complained to you before That the way block chains work today is Somebody post a transaction one miner Verifies it post it on a blockchain but Then all the other miners have to verify It as well it’s a lot of wasted work for All the other miners you see everybody’s Very fine at transaction so roll up says Let’s get rid of that let’s make things Way more efficient and so we introduced A central server it’s called a rollup Server but of course anyone can run a Roll-up server so it’s in central Centralized but it could be arbitrarily Replicated and then inst instead of Sending their transactions to the to the Miners all the users are going to send The transactions to the roll-up server What the roll-up server is gonna do is He’s gonna verify that each transactions Are valid so he’s gonna verify all the Signatures verify all the account
Balances everything is going to be all The checks are going to be done over in The roll-up server and then what the Roll-up server is gonna do is he’s gonna Generate a snark proof that all the Transactions are valid okay so this Proof pi yap i stands for proof is a is Fast to generate It’s convincing and it’s super fast to Verify yeah so you can roll up a Thousand transactions into a single Proof and verifying that single proof Takes like five milliseconds yeah so all The miners have to do now is just verify Proofs rather than verify transactions So this guy does all the work and Everybody else benefits from it yeah That’s the idea of roll-up this is an Amazing way to scale a blockchain Because here you can actually have like A thousand transactions possible even Ten thousand transactions rolled up into A single message onto the blockchain and By the way it’s called roll-up because You also send the summary of the Transactions on to the blockchain so That gets written the summary gets Written to the blockchain so if somebody Wants to they can start at the beginning Of time and kind of go through all the Roll-up transactions and rebuild for Themselves the current state of the World yeah this this is kind of Important this is what this is a primary
Characteristic of roll-up that the Actual transaction data gets written to The blockchain just nobody needs to Verify this data because the proof Proves that it’s legitimate okay so That’s that’s roll-up beautiful idea for For for a snark you realize we didn’t Need the zero knowledge property here All we were doing is that all we were Using is the property that these proofs Are efficient to verify there was no Need for zero knowledge people still do Zero by the way it’s called a ZK roll-up People still do zero and all these Roll-ups Just to make sure to provide some Privacy so that the miners don’t even Know what the transactions are so we can Even do provide some level of privacy From the miners but not privacy from the From the roll-up server turns out with a Bit more work you can even provide Privacy from the roll-up server but That’s that’s for another day so that’s One application the other application I Wanted to mention is one that’s right Now is a bit of a dream but this is what We’re working towards okay this is a Still science-fiction but we’ll get There yeah we’ll get there for sure and This is again really important to Remember especially in the context of The all these central bank digital Currencies yeah it’s really really
Important to remember that you can have Private data on a public blockchain yeah Let me explain why this is so important So today essentially all the data is Supposed to the blockchain is available In the All these transactions are available in The clear this basically means that it You know it’s not usable for many b2b Applications for example you know if Apple wants to pay Foxconn in you know Digital currency everybody could look on The blockchain and just see exactly how Much Apple is paying Foxconn which is Not something Apple being Apple would Want public yeah What’s important to remember is that you Can actually this is not necessary you Can have private data on a public Blockchain so let me explain what I mean By that so rather than writing the Application code in the clear and the Transact in the application state in the Clear what we can do is we can write Commitments to the application code in Commitments to the states on to the Blockchain so we’re committed to what The data is to what the application code Is it’s just nobody knows what the Application code is it’s inside of a Hiding commitment okay and then when a Transaction happens when we need to move To a new state so here you can see the Transaction is hidden we only post
Commitments to the transaction on to the Blockchain and we post posted Commitments to the updated states what We do is we also update a zero knowledge Proof that says that this state Transition is valid state transition is Done according to the committed code ok So just again this is a little bit mind Boggling that this is at all possible It’s a kind of an amazing idea that you Can have secret code secret state and You can have you know updated secret States and even though nothing is public Everything is hidden you can provide a Zero knowledge proof that the code was Applied correctly to the state and the Transition is valid yeah so you have This property called public Verifiability where anyone in the world Can verify that all the rules are being Followed correctly the code is running Correctly even though nobody has any Idea what the code does and nobody has Any idea what the data is and yet you Can verify that everything was done Correctly it’s kind of magical yeah that It can be done but it’s it can’t be done I’m telling you it can be done and we I’m happy to actually explain how it’s Done but we would take we would have to Be here for another three weeks it’s Actually really pretty it’s really Really pretty I would actually be Excited to tell you that’s it’s really
Cool Yeah I don’t know it’s it’s what can I Say it’s just tricks with polynomials Yeah if you like polynomials you know go Look at how this stuff works because It’s really really pretty how it works Just tricks with polynomials okay so Then basically all these transactions Sort of happen the state evolves and Every time we have a state evolution we Just provide a zero knowledge proof Everything is verifiable but the data is Completely hidden yeah so if anybody Tells you We’re gonna be running a private Blockchain because we’re worried about Privacy you tell them that’s the wrong Solution you should be putting your data On a public blockchain and just use your Knowledge proofs yeah that’s much better It’s better for interoperability it’s Better for public verifiability and it’s Just better all around all right so That’s the whirlwind tour of the crypto Primitives I wanted to tell you about so One thing that I want you to remember Kind of drives me nuts sometimes I talk To folks who tell me we have some data So we’re just gonna throw it on a Blockchain yeah that’s not what a Blockchain is for yes it’s not a Database yeah just remember a blockchain Is not a database basically the question Is always to ask whenever you come up
With an application you think you have a Really cool application for a blockchain The first question you need to ask is Why can’t I just use a centralized System centralized systems are a million Times easier to build than a Decentralized ones there are a million Times faster if you can centralized Centralized yeah it’s just when you Cannot centralized that’s when you need Centralized so basically if there’s no Single party that’s trusted by it by Everyone that’s where you have to to Decentralize but of course you get Complexity and and and speed issues as a Result yeah so that’s one point to Remember it’s not that everyone who has Data needs to put that data on a Blockchain that’s not what quite what It’s for the second thought I want I Want to leave you with is you guys Remember in the early days of social Networking there were like a bazillion Social networks out there yeah and what Happened at the end yeah what happened At the end was that we were left with a Relatively small number right now we’re Kind of at the age where people are Experimenting with lots and lots and Lots of blockchain architectures yeah so Right now we’re kind of at the age where There are lots and lots and lots of Architectures out there and well you Know maybe there will still be many
Blockages out there in in many years but It’s very like Will converge on to a small number of Architectures yeah and so which ones Those are I wish I could tell you it’s Too early to tell but yeah this is why It’s so much fun and you know I Encourage you to kind of play with these With these ideas and who knows maybe Your design will be the one that gets Chosen and that’s what the world will Converge to okay fantastic so I think I’ll stop here I’m really looking forward to seeing Everything that you build and yeah thank You very much for the roll-ups This server that’s hosting the actual The code and the algorithms for it is That a centralized server so the server That’s actually building the proofs That’s a centralized server but it can Be replicated so in other words the fear Is that you might have censorship right I mean the server might the server can’t Cheat you because it’s providing a proof That everything that it does is valid And correct what the server might do is It might refuse a transaction from you Yeah a censorship so the way the way Around that is well there could be Multiple servers multiple entities that Provide roll-up services and you could Choose any one you want They don’t need to talk to one another
So you come to consensus through the Layer one blockchain right so they push These hashes to the layer 1 and then the Layer 1 basically enforces consensus if Anyone tries to let’s say if two of them Tried to post transactions that are Conflicting the layer 1 basically will Resolve will choose one of them and We’ll go with that and the other one Will get will get rejected because the Zero zero zero knowledge proof will no Longer verify yeah yeah yeah it’s kind Of a pretty cool design yep Yeah that use snarks could you recommend Libraries that abstract away the Underlying complaints yeah there are Many little primitives look so there’s a Like I said we use circle because it’s The easiest one to come up to speed with So we don’t spend up to spend a lot of Time learning that but there’s no Parties there’s a snarky there’s Jason Arkham it is now there’s Inc which Is which is kind of a cool with zinc is One that you can write and rust you Literally write your program in rust and Zinc will compile that into a snark yeah It’s just yeah a circle is the easiest One to learn but there are lots and lots Of options to choose from yeah Thank you yeah yeah when you know got to The slide about private data on a public Change you said this is still a dream ah Why why is that still a dream gotta
React like what needs to be built yeah Good good that’s a really good question So it’s it’s actually we’re almost there So so if all you if all you want to do Are just simple simple funds asset Transfer transactions that we can Totally do today yeah in fact that Exists today already if you if you want To do more complicated transactions like You want to run the entire an entire EVM Program inside of a snark you know That’s that’s gonna take a while so Generating the proof the proofs will Just take take a while The nice thing the good thing though is These proof systems are getting better And better and better in the sense that The bottleneck the bottleneck right now Is the time to generate the zero Knowledge proof that overhead keeps Shrinking yeah we have better and better Systems and yeah so basically the the Time to generate the proof keeps Shrinking because the algorithms get Better and eventually the hope is that We’ll get to a point where we can run Arbitrary complex program programs Inside of a snorkel we’re not yet we’re Not there yet today but then at the same Time you know most transactions are just Asset transfers and those we can handle Already using the existing existing Technology so we’ll get there I’m I’m Confident we’ll get there so that’s that
That’s the goal this is by the way why It’s so much fun right I mean this is Kind of you know I’m a cryptographer for Me you kind of dangling you know it’s Like a red cloth dangling in front of my Face saying you know bill dusty snarks In it’s like I can’t believe you guys Are asking me to do this is like my Dream thing that I like to think about So this is why it’s so much fun yeah Yeah you emphasize in your talk Importance of not putting data on the Blockchain How you advise to go around data Availability problems Like currently for instance that you put Data not in the state but in the first Ask all data are there any but more Sophisticated ways of doing this and Yeah so the current roll-up solutions They just put the state put all the data And call the call data so at least it’s Somewhat compressed but there are other Other proposals out there so for example One one idea is you have a committee That replicates the data and then you Declare that the data is available if All the committee members sign yeah so You’re kind of relying on a hundred Party 100 parties basically to provide Data availability for you yeah so Instead of pushing everything to the Blockchain You just have to push these hundred
Signatures to the blockchain and then we Say that data availability has been done And if you need to date the data Presumably you go to any one of those Hundreds or a subset of those hundreds And you can recover the data so that’s Like that’s a pretty promising fairly Promising technique but today basically Call data pushing call data to the Blockchain is basically what what you Have to do with roll-up yeah so yeah so You’re right this is this is expensive And over time my guess is we’re gonna Move to these companies yeah it’s good Good question yeah so this action might Be related to that question but so Recently on aetherium someone had Written an obituary for the Chinese Scientist who had I guess raised the Alarms for coronavirus and in that case You know data is being written onto the Chain and maybe the goal there was to Avoid censorship by the Chinese Government so I guess I mean given that And your comment about keeping data off Of the chain that’s a really good Question so if you if you had so could You use the chain as a way to just Prevent it from being removed yeah Absolutely actually there in fact there Are a number of projects that do exactly That they use they use a blockchain for For replication in fact that’s kind of What file coin is
Is doing so the data can never be Removed so those I would say those are Chains that are specifically designed For data storage so they’re the cost of Storing data on the chain is not as Expensive and that that is a wonderful Application for that yeah so you put the Data they’re like on file coin and or Any one of these other systems and Because of replication it’s very Difficult to get to make the data Disappear so yeah that that’s a perfect Application but those are specific block Chains that are designed for that Purpose so it’s a really good point Thanks I’ve met a lot of VCS that talk About the deals that they’ve passed on That became billion dollar companies and Bit corners that talk about the keys That they forgot on that old laptop and Destroyed I find I worry a lot about key Management for my users is there a way To do it without doing custody Oh boy you that’s a difficult one haha Right so yeah so there are there are Well what can I say so there are some Proposals well there look there are many Proposals to try to address exactly that So there are there are some proposals Proposals to try to address this using Using Hardware enclaves so you do Custody to a remote party but the Custody lives in a hardware and clave so They can’t abuse it unless you authorize
It right so there are there are ideas Floating around but yeah but what can I Say you’re right I mean keeping keys Yourself even though it gives you the Comfort that no one can take can take The assets away there’s also a risk that You might lose your keys though Obviously as you know there are Combinations of the two right where you Keep at will you do a two or three Secret sharing where you know you keep One share and there are two servers to Keep the other other shares so even if You lose your data maybe you can use Those two servers to recover your to Recover the assets nevertheless so there Are solutions so I would call that Partial custody right so there are lots Of people exploring different ways to do Partial custody Yeah and that that’s quite popular of Course full custody is Of popular very popular as well then you Know I have to say the nice thing about This world the world of these assets Digital assets is that the user has a Choice right today with the financial World today you have no choice right you Know if you think about your car Ownership slip your your housing house Ownership all that kind of lives in a Custody based databases you have no Choice that’s basically how the world Runs the nice thing about this is you
Can choose right if you are you know if You if you are worried about custody and You want to keep the data for yourself You can just keep the keys yourself There is risks to that but if that’s What you want you can do that if you Want to have a custody solution you can Do that too if you want partial custody You can do that too so in some sense It’s kind of nice that we can service More of you know more more people’s you Know needs and you know give them Whichever level of comfort comfort they Want so I think that’s actually pretty Powerful capability that’s not available Today okay so thanks Dan yeah thank you Very much [Applause] [Music] You [Music]
