An EOS Smart Contract for Block Producer Information
There is a long road ahead of us on EOS with both developing and enforcing Block Producer standards. However, one improvement that many of us can agree on right now, is that it's time we all started putting our information on-chain, rather than only on our websites.
Currently, most Block Producers are following a standard "bp.json" format (see here), to explain extended information about their candidacy. For example, you can see ours here. However, with this information only being on a candidates website, we run the risk of issues such as public defacing, DNS changes, non-verifiability, and more.
The solution to these issues is simple: store this information on-chain, signed by the producer themselves. Doing so brings multiple new benefits as well: we can have dApps parse all block producer candidates' json files, we will have a verifiable history of changes, and finally we allow validators to have a single consistent place to look up all candidates' publicly posted information (rather than scraping individual websites).
In order to help facilitate the transition to storing this information on chain, we have created, purchased RAM for, and deployed a new smart contract producerjson that enables this. We now open it for peer review, feedback and suggestions, and if desired -- immediate use.
See The Contract
The contract code is open source here:
https://github.com/greymass/producerjson
As you will see, the code itself is fairly simple -- it does not attempt to validate the contents of the producer's json at this time. This choice was made in order to allow external validators to be as strict or relaxed as they choose to be when grading a given producers information, as well as allow quick changes to the BP standard format without a contract update.
Try It Out
Pushing your bp.json file on chain is a simple two step process. Just grab your bp.json file, and push it to the blockchain with a cleos command like our example:
cleos push action producerjson set '{"owner":"teamgreymass", "json": "'`printf %q $(cat bp.json | tr -d "\r")`'"}' -p teamgreymass@active
Here's the result of that action, on the blockchain.
As you can see, your information will be published to the chain in the producerjson table right away. You can see yours and everyone else's information with one simple command:
cleos get table producerjson producerjson producerjson
It takes about 1.5 kB of RAM to store a bp.json, which is not expensive at all -- this is less than creating a new account. Due to it being so cheap and easy, we hope everyone will join us in putting their information on-chain, organized and verifiable for the public to see.
Future Contract Changes
If this standard takes off, we are happy to convert this contract account into a multi-sig account and allow shared control with other top Block Producers. This would slow down any required changes to the contract, but also ensure the longevity of it.
Any feedback is welcome, and we'd love to see any recommendations on how to modify the contract. Feel free to comment below, or contact us on telegram!
This project is an effort lead by Team Greymass, a Block Producer candidate for EOS. If you like what we're doing, don't forget to vote for teamgreymass on EOS!
From our Telegram chat:
Luke Stokes, [Jul 20, 2018 at 12:30:20 PM]:
Looking at this and discussing it now. Curious your thoughts on having a seperate table just for endpoint discovery? I'm getting a little pushback in terms of the workflow for a wallet to download all the data just to get the endpoints is a bit much. We created this file (updated every 10 minutes) to help: https://eosdac.io/topnodes.json
but I'm curious your thoughts on using multiple tables with straight data instead of just one json file
we also craated https://eosdac.io/topbps.json (also updated every 10 minutes)
on chain is a much better approach, for sure, but having everything in json may not make sense if someone just wants the nodes
Scott (anyx), [Jul 20, 2018 at 12:32:15 PM]:
Not really sure a seperate table is needed; you could parse the json from the producerjson table and grab as many endpoints as you need. 🙂
I briefly explained in the post why I don't think mutliple tables is the best idea; the BP standard is likely to change over time (possible rapidly), and validation should be clientside.
Luke Stokes, [Jul 20, 2018 at 12:34:06 PM]:
I only see this:
But not much mention of the downsides of multiple tables (one for nodes, contact info, etc)
Maybe staying in json for now makes sense to keep a fluid, dyanmic standard as it's developed, but more and more I'm thinking we should be taking RFC approaches to do the hard work up front of defining a set standard
instead of assuming it will change later (and thus lead to breaking apps)
Scott (anyx), [Jul 20, 2018 at 12:36:20 PM]:
The big downside for multiple tables (at this current time) is the standardization efforts. Right now it just takes a json blob, and external validators can determine the convention.
In my opinion the transition to this will be much easier than conforming to a new standard, and we can change things over time if desired.
It's literally a one-liner cleos command to put your bp.json file into this table, so hopefully this gets traction.
Luke Stokes, [Jul 20, 2018 at 12:38:30 PM]:
What about the costs associated with wallets and apps having to change things, potentially multiple times? If they are pulling json files now, then they change to pull on chain then later they change to pull from individual tables once a standard is defined. In the meantime, how do we decomission old, legacy approaches? that could prove to be quite difficult. There is a cost associated with supporting approaches now which may become legacy in the future.
Scott (anyx), [Jul 20, 2018 at 12:39:08 PM]:
This is true regardless of using seperate tables while the standard is changing
Luke Stokes, [Jul 20, 2018 at 12:41:37 PM]:
I think that's kind of my point: Wouldn't it be better to do the work of figuring out a standard first? That way we have fewer changes all around and we remove one change in this cycle: pull json from web, pull json from chain, pull data you want from chain
(btw, I'm partially pushing back just based on push back I got from my own team. I think your idea is a great improvement on what we're currently doing)
I didn't see this conversations, but my opinion is that this contract isn't for node discovery.
Node discovery itself shouldn't even be on chain - since you'd need to connect to the chain to even access that data. It's a bit redundant, and if you're already connected, why would you need to discover more nodes?
This contract itself I feel is just about publishing the bp.json data on-chain - so that way the many wallets and web apps who need this information don't have to scrape 100's of websites.
I agree scraping 100's of websites doesn't make sense, but the pushback I'm getting from my team relates to the lack of a clear standard and the potential to have to support every backward compatible solution we introduce from here until whenever. If more people use this, will they also continue to support the website based bp.json file? If a full standard is developed in the future which puts this data directly into tables instead of as json, will that then be a third approach (with two legacy approaches to support)? That, I think, is the primary pushback I'm hearing from my team. A desire to plan out good specifications first, before introducing temporary solutions which will then need to be supported in order to not break applications which start to rely on them.
Unless we think the bp.json approach (both on the web and stored on EOS) is the solution the EOS community should be using for quite some time? I mostly agree with you that this is a good improvement on what is being done now, but I'm doing my best to relay concerns my team has to hopefully find some common ground. Maybe it's okay that app developers just have to change things as older methods go away naturally, but I think some prefer more structured, planned approaches and assurances that when we introduce a new approach, we've thought through if this is the best solution for the problem we're solving.
If this isn't for node discovery, then the current bp.json hosted on websites were were provided via listproducers still seems to be needed. If that's the case, is this data duplication?
This solution is both backwards compatible and forwards compatible. Nothing needs to change from a validator or info collectors' perspective aside from, instead of retrieving the data from 100's of websites, they only need to grab it from the blockchain in one call. There's no "legacy" support needed, so that makes zero sense, as the standard has not changed -- only the location of storing it. App developers do not need to change anything with regards to parsing the data.
I genuinely can't understand why you want to push back against storing this information on-chain. Why wait? Why do you want to keep using unverifiable websites? This is completely unlike you, so I'm curious what the real agenda is with that. Is it because your team spent time developing tools to scrape websites, and that work is about to be outdated? If that's the reason, then honestly, you need to really think if that's best for the EOS network or what your true intentions are.
No, it has nothing to do with anything we've built at all. I only want what's best for EOS.
I completely agree storing on chain makes more sense than storing on individual websites for a number of reasons. Ask @jesta to share the chat I had with him yesterday for more details. I'm kind of stuck in the middle a bit because I agree what you are doing is a worthwhile improvement, but I'm working through concerns of my own team, so I'm bringing those concerns as questions to you (even if they aren't ones I personally, passionately care much about).
I've spent way too much time arguing both sides of this (it really has become a bike shed). I'm happy to put our data on there and plan to do so (even if some on my team aren't excited about the idea). I think they prefer going from bp.json on websites straight to a well-defined, on chain standard. If app developers switch locations from website to onchain, that is a coding change which they will have to make and if they are already making that change, then the thinking here is should think about if there's a better, well-designed standard and on-chain approach beyond just a blob of json. Again, I'm not too concerned about it personally, but the idea is that some temporary fixes end up sticking around for a long time and then become difficult to remove in the future if too many systems are relying on the legacy approach. Removing the legacy approach later then breaks those applications. If we can void adding something we know will be a legacy approach soon, that's a good thing. It might be best to do it right the first time instead of having an intermediary approach. Unless we think this JSON is the best way to do it?
Maybe Michael Yates, You, Aaron, and myself can jump on a call soon to work out a standard we can propose to the community so we don't end up supporting two things on chain and instead only have one thing on chain. If we think storing this data as json on chain is the best solution and what we will do so for years to come, then it's great and we can just use this. If we think there's a better way to do it within EOS, maybe we can come up with that fairly quickly and do that instead.
I still don't understand it. This isn't a legacy thing. The "legacy format" is the current format, a JSON blob. This is what is on the websites right now, and this is what will be in the table.
There is NO CHANGE TO THE FORMAT. There is no legacy left behind. This is not a "temporary fix". Applications dependent on scraping websites can move over to this single blockchain call. There is no "legacy system" as the system is the same format, the only thing that has changed is the location of the data.
Why are you really pushing back against putting it on chain? The standard is probably never going to be well defined, and will be in flux for a long time -- and then, since that flux might never stop, we'll never move off using websites, while we squabble about standards forever.
When we move over to a standard with multiple tables (IF we can somehow fix the standard and get it out of flux), then we will have to change all tools for a format change. This format change can come later, because, once again. There is no format change with this contract. Only a data location change to on-chain.
Validation tools do not require any format changes! It only gets rid of the need for website scraping tools!
It's time we actualize "Blocks or it didn't happen".
This API gathering script you have, which requires looking through BP's websites, scraping their BP.json files (if they exist!), and collecting all that information, finding which API nodes exist... is now replaced with a single command.
I've replaced the whole gathering process with a single call to the table, like so:
Tada! Output looks like
As right now only 2 BPCs have plugged in their bp info's. But as BP's add their data, this call will always get the latest info.
Now anyone can validate immediately and not require any websites, just the blockchain itself.
+1 Thanks for sharing anyx! Very elegant way to retrieve SSL endpoints:
Notice there's 3 endpoints now wink (EOS Nation the 3rd producer to register)
We have now done it.. But I do not see swedencornet by ./cleos -u https://eos.greymass.com get table producerjson producerjson producerjson
Ref: https://bloks.io/transaction/522eea0fa90bd9e9ea3f97189d32fddddd3ffb57ea9617a0e1f746021bc9b501
If the end of the json contains "more" : true , you need to use limits and pagination to get all the results.
Thanks! Done it now...
Great job Greymass! This is a great approach to store Block Producer bp.json on chain. Looking forward to see more Block Producers host their bp.json on chain!
Thanks for that! We thought so too, and it seemed like a low hanging fruit we could try to knock out for the entire community. Hopefully we'll start seeing some user interfaces adopt it's usage :)
Are we just hosting bp.json on chain? Or how does it ensures the validity and correctness of data provided in bp.json?
I fully support this initiative. We should store BP information on Blockchain instead of relying on off chain web sites.
My only feedback is to add JSON validation to smart contract (if possible) to make sure we are storing valid data on Blockchain.
@EOSTribe published it's bp.json on Blockchain as well!
This is EOSTribe... What exactly is not matching? I also have similer error:
https://validate.eosnation.io/producers/eostribeprod.html
A really good idea and it works perfectly, good job. I've added the @blockgenesys bp.json file to the table.
sweet initiative. tip of the hat from EOS Romania.
thanks to this eosromania22 bp.json is now on chain
This is huge. The EOS community has been waiting for something like this. Thank you for putting this together. Greymass rocks!
Yes, it is great work.. But validation has to be an independent process...
Done! Please let us know if something is wrong?
https://bloks.io/transaction/522eea0fa90bd9e9ea3f97189d32fddddd3ffb57ea9617a0e1f746021bc9b501
Hello you are keep changing this with high pace... And I understand that it takes time to mature... Security is a continuous improvement process. But are you involving all block producers or just few hands are involved in changing parameters and validation process... We are already too afraid to give lot of power to top 21 block producers ad we hope that their should be INDEPENDENT VALIDATION process.... and Future Contract Changes could be managed by INDEPENDENT VALIDATORS, NOT by top 21 BPs..
This works especially well on mobile, one call to chain/get_producers and another call to getTableRows->producerjson and we have a filtered list of eligible block producers and their information.