Celo Discord Validator Digest #15
Expansion of the validator set, attestation errors and sender ID pre-registration, oracle circuit break, useful info and community fund.
One of the troubles we have faced as a validator for Celo is keeping up with all the information that comes up in the Celo's Discord discussions. This is especially true for smaller validators whose portfolios include several networks. To help everyone stay in touch with what is going on the Celo validator scene and contribute to the validator and broader Celo community, we have decided to publish the Celo Discord Validator Digest. Here are the notes for the period of 17-30 August 2020.
Discussions
Expansion of the validator set
With the election threshold keep on growing, zviad | WOTrust | celovote.com noted that maybe it was time to start discussing expansion of the validator set:
... I would like to float around the idea of increasing the total number of validators once again.
Current minimum to elect a single validator is at 1.9 million CELO, which already seems very high, and in addition to that, there seems to be no stopping to it continuously going up. Considering locked gold is at 328 million, hypothetically it can go up as high as 3 million per validator.
Positives of increasing total number of validators:
Less incumbency bias and potentially more decentralization due to lower cost of entry.
More income for core foundation voted validators who should be able to elect 2 validators once again. I think more income will continue to incentivize those validators to continue to stay engaged and keep community active. (there is a pretty big difference between 2 vs 1 elected validators in income).
Shifting more of the new CELO inflation to validator rewards vs voter rewards. I think this could be both a positive and a negative because it should further encourage a bit more CELO liquidity instead of all of it being locked.
Cons:
Potential perf and stability issues
Decreased voter rewards (but again this can be considered a positive too)
As a practical proposal, I would propose going pretty conservative and to add 10 new validator slots per month for next 5 months. So in the end it would go from 100 -> 150 over course of 5 months.
@syncnode (George Bunea): I support what you say about adding more validators. However my stake is very small and the voting power is centralized at the moment in the hands of very very few actors, so unless this idea gets some serious support for important voting power holders it’s going to be pretty hard to see it pass. I would suggest to open a forum post and discuss there pros and cons.
Attestation errors and sender ID pre-registration
Many validators see missing attestation requests even though their attestation services seem to work properly. warfollowsme | Celomap.io provided some additional insights about the attestation requests prompting further discussions:
Hi everyone. We've same issues with attestations. To figure out what happen I started looking at the history of all the attestations. There's a lot of interesting things. For example, most attestation requests are sent to multiple validators. And there are requests that all are not completed, and there are cases when someone successfully confirmed the request and someone did not. Therefore, the moment regarding the countries whitelist either does not always work or the attestation nodes has different configuration.
Since you can't see the history of attestations in any explorer for now, I created a temporary solution in the Google sheet. https://docs.google.com/spreadsheets/d/1HtZCJvOs6cktgT7-gA_LdAFfRbSbndUmKb_qlpJqTAI
btw to check coming attestations in @Celo_Ryabina_bot for your validators, you can subscribe on
AttestationIssuerSelected
andAttestationCompleted
with issuer filterAdvanced - Attestations - AttestationIssuerSelected - Add filter - issuer - enter your validator address - Exit filtering mode - Create NotificationAdvanced - Attestations - AttestationCompleted - Add filter - issuer - enter your validator address - Exit filtering mode - Create Notification
And then if you get
AttestationIssuerSelected
message and after 2-5 min you receivedAttestationCompleted
that mean all good and attestation is confirmed, but ifAttestationCompleted
didn't come, you missed one.@Thylacine | PretoriaResearchLab: Is there a potential Sybil/social attack vector here? Like if someone is using a device emulator with the APK and automating actions?
Make up a valid but undeliverable mobile phone number.
Spam account creation process and lower existing validator set's attestation scores, making them appear unreliable to the Foundation.
???
Profit?
@warfollowsme | Celomap.io: First 2-3 are free and then set a fee that will increase with more requests.
@zviad | WOTrust | celovote.com: I think there is easier accounting for it. Since at least three validators send the message, if none of them are able to deliver it, then it is more likely that something is wrong with the client than any of the validators.
It is reasonably easy to pull up stats only for clients that actually succeed in validation at the end, and missed stats from those can be more important to track.
@warfollowsme | Celomap.io: Just want to clarify, in the criteria of the Celo Foundation Vote specified
Commitment to running attestation services with demonstrated reliability and uptime
Such mistakes, or situations with roaming, are not considered errors of attestation service, because it is not its responsibility. Is it correct?
@Thylacine | PretoriaResearchLab: This is exactly what I'm concerned about. I'm running the service with full uptime but still getting failed attestations essentially because of the luck of the draw. I have worldwide settings on, everything working fine.
Perhaps we need a second authentication method, kind of like the WeChat social sign-up method? (You can be invited by someone who already has an account by scanning their QR code). The attestation issues might genuinely become a problem in say, Africa, where people might be on roaming towers constantly in rural areas, or have less reliable cell infrastructure?
To mitigate the issue, cLabs released a new attestation service image that allows validators to use both Twilio and Nexmo as sms service providers so that in case one of them fails, the second provider will be used to send a message. Validators were asked to upgrade to ensure that no attestation requests are being messed. Generally, version 1.0.2 is fine, however if you are using a proxy (like Cloudflare) in your setup, you should be using the following image:
@timmoreton | cLabs: ... Try:
us.gcr.io/celo-testnet/celo-monorepo@sha256:6ffd6e25c5e14a9a84c6f3d31ba536ff8ff7c761c596fb48c7348a7a78c4339b
which tag isattestation-service-645ea83bcb53f0ef53b14ac9312d7a77913c9754
Oracle circuit break
Following the announcement of CELO being listed on Coinbase Pro, the price of the native asset jumped so high very quickly so that the on-chain oracles didn't update the prices for some time:
@zviad | WOTrust | celovote.com: Today was certainly exciting day for oracles and Celo exchange. I am guessing due to this announcement: https://blog.coinbase.com/celo-celo-is-launching-on-coinbase-pro-c69ff2690691 price on exchanges jumped so high that oracles didn't update prices anymore. And now due to arbitrageurs onchain price is being brought to market price at a steady pace. Arbitrageours are low-to-medium volume, but still pretty decent. Pushing ~100k cUSD->CELO volume in ~2hrs time.
@trevor | cLabs: Haha yeah, the first test of the circuit breaker! Happy to see everything went as expected. A number of cLabs protocol economics folks hopped onto a "war room" meeting t5o track what was happening and determine when it felt safe to let the oracles report again. After making sure things felt stable wrt spread, volume, prices, etc. we enabled them 10-15 minutes ago. Even without bucket updates the on chain exchange seemed to behave as expected.
@hqueue | DSRV | CeloWhale.com: ... Yeah, it seems oracle circuit break worked about for 94 mins. According to our monitoring system, there is no oracle report between 2020-08-26 15:00:17 (rate 2.933) and 2020-08-26 16:44:17 (rate 4.1625).
@trevor | cLabs: Yeah the circuit breaker recovery is manual right now. Interesting idea about being close to the market price though.
I think it's just very hard to codify when market conditions feel healthy enough to close the circuit breaker again, so keeping it manual feels like a more conservative approach.
@zviad | WOTrust | celovote.com: I think manual recovery makes sense, as long as procedure for activating is well documented and well understood. (i.e. what kind of permission is needed by whom to press that recovery button)
Even if it is manual, more standardized it is the better, it will only make it easier to make that part also decentralized or automated in future.
@Patrick | Validator.Capital: So what happened during that period of time that there were no oracle reports? Did the CELO buckets get depleted?
@trevor | cLabs: Trading with the on-chain exchange still is possible, but the buckets just never get reset. What happened is it pretty much behaved the same as normal Uniswap on Ethereum, where there's an arb opportunity to trade one direction on-chain which slowly brings the on-chain exchange rate toward the "real" market exchange rate.
On https://thecelo.com/ you can see some cUSD -> CELO trades that slowly increased the on chain exchange rate because of an arb bot.
So the reserve lost a little, but not a lot.
Useful info
A note from zviad | WOTrust | celovote.com for those trying to deposit cUSD to OkCoin:
Just a heads up for anyone trying to deposit cUSD to OkCoin. It seems like something is broken with their deposit system and funds aren't being properly deposited. (It might be an issue with just our OkCoin deposit address, but that seems unlikely). Just be aware that if you deposit cUSD right now you might have to reach out to their support to actually get your funds deposited (and it will take few business days at least before you are able to access them).
Community
You might be unaware of the fact, but there's a community fund, and any community member can submit a funding request:
@victor | cLabs: One of the things we were chatting about in the Validator happy hour is that the Governance controlled community fund (i.e. the balance of the Governance smart contract, which may be spent via Governance) has been accumulating is a currently a at the sizable sum of ~1.75M CELO (= $12.6M @ $4.12 / CELO) Anyone can propose a funding request, or some other idea of how to utilize this fund toward the benefit of the community, and it's up the CELO holders to grant it. There is not really a process around this yet, but it seems to me that the first step is to raise awareness so everyone can discuss and agree on what such a request might look like.
A feature request I have for @Shen | Bi23 Labs | thecelo.com @Emma |Bi23 Labs | thecelo.com is to show the Governance balance on thecelo.com.
(If you are curious about getting it right now, something like the following works)
const { newKit } = require('@celo/contractkit')const BigNumber = require('bignumber.js')const kit = newKit('https://rc1-forno.celo-testnet.org')const governance = await kit.contracts.getGovernance()const balance = new BigNumber(await kit.web3.eth.getBalance(governance.contract.options.address))console.log(balance.shiftedBy(-18).toString())
Like what we do? Support our validator group by voting for it!