Transcript for #bitcoin-dev 2017/12/26

02:51 vinnix (bitcoind running with valgrind result)
07:29 weiyang I am trying to use curl for experience the rpc method. Not sure which user/password should I try.
07:29 weiyang
07:29 weiyang
07:41 DSidH question re. segwit: "For receiving payments, the wallet must be able to create a P2SH address based on a P2WPKH script (defined hereinafter), and be able to recognize payment to such addresses."
07:41 DSidH from the docs.
07:42 DSidH what does it mean "able to recognize payment to such addresses." ?
07:58 arubi able to watch the p2sh scriptpubkey for incoming payments and then solve the p2sh and p2wpkh to redeem
07:59 arubi a lot of wallets watch just one script type for a pubkey. core watches a bunch by default and can be taught to watch more
12:16 vinnix it seems like leveldb has some memory leaks
12:16 vinnix at least that is what I am observing when using "-reindex" (after reindexed and reading again all blocks)
12:17 vinnix (bitcoind running with valgrind result)
19:00 DSidH arubi "a lot of wallets watch just one script type for a pubkey. core watches a bunch by default and can be taught to watch more"
19:01 DSidH that would make sense if the only payment it expects is of that script type. So each wallet should watch for payments to all addresses it owns
19:36 DSidH arubi nvm I understand, someone can send to my P2PKH address once he knows my public key without me ever giving them the address, so I need to watch all address types
19:40 arubi ah just caught the highlight. it all depends on how many scriptpubkey programs the wallet can solve by itself right
19:42 arubi so yes knowledge of a bare pubkey allows some person to send you coins to whichever script, but mostly you'll watch the standard scripts. core can pick up on bare scriptpubkeys where it knows the private key, so something like a 1-of-1 multisig with a pubkey of your would be picked up
19:43 arubi maybe it can pick up on multisigs with more keys if it knows privkeys for all pubkeys used, actually I haven't tried
19:45 arubi basically you wouldn't expect someone to just randomly pay an address or script you didn't specifically give out to them, but knowing to watch for and solve the scripts is quite useful
20:05 buffalobill33 If I want to write a bitcoin payment system for a site, what is the best way of doing it? Should I write my own wrapper for bitcoin RPC calls?
20:07 arubi you could. just generate a huge keypool in the wallet first then call getnewaddress for payment addresses
20:07 arubi but it's not easily used for multiple customers without more tools, maybe for donation addresses that's enough
20:08 buffalobill33 what would be the suggested way of designing such a system? One that could scale to say tens of thousands of users easily
20:09 arubi not sure, sounds out of my reach
20:09 arubi I mean, it sounds like a full fledged enterprise product right
20:14 buffalobill33 The end product would be. The reason I ask is because experienced bitcoin developers with availability are hard to find, and I'm wondering how difficult it would be to develop the experience myself
20:16 Randolf buffalobill33: Well, you'll obviously need a good database to keep track of your users, and excellent security practices for handling passwords, 2FA, etc. Fortunately there are many high quality, free, and open source databases you could use (e.g., is my favourite).
20:17 Randolf buffalobill33: You'd also need someone to develop the software for you. There are many options, and you can combine multiple languages together, but it would be smart to choose a language that your web server software has good support for (e.g., with Apache HTTPd there's mod_perl2, Python, Java
20:17 Randolf Server Pages, PHP, etc.).
20:18 Randolf buffalobill33: I agree with arubi that you'd be developing something in the realm of a full-fledged enterprise product.
20:18 Randolf buffalobill33: If I were to embark on such a project, I would hire a team of good developers to build it in-house.
20:19 arubi and these are just the things that you'll need for just any major online service. add cryptocurrency to that and you have all the issues that come with that
20:20 arubi mostly you'll see that no matter what technology services build on, they always end up getting hacked or lose a bunch of coins somehow
20:20 buffalobill33 Randolf: I have most of the backend developed (rails with postgres) with security features such as 2FA, tokens etc in addition to distributed locks and rabbitmq as a messaging system. I am looking to get people to help me develop the bitcoin functionality and want to get a rough idea of how much work needs to be done and how difficult it is conceptually
20:23 buffalobill33 I plan on having a full third-party security audit at the end. How difficult is it for an experience programmer with a reasonably good conceptual knowledge of bitcoin to develop a backend api to process payments securely?
20:24 arubi maybe look into btcpay, it's software for transaction clearance
20:26 arubi many different results on google, is what I mean
20:26 buffalobill33 thanks, having a look now
20:30 DSidH [01:13] <arubi> maybe it can pick up on multisigs with more keys if it knows privkeys for all pubkeys used, actually I haven't tried
20:30 DSidH Hmmm I dont think its possible because core would need to know all pub keys used in the script.. else it won't know
20:30 buffalobill33 Right now I have an experience blockchain dev with a small team offering to build a btc and ethereum payment processing backend for $70k. Is this a reasonable price? How long would such a project normally take a team to do
20:31 DSidH oops u said all
20:31 DSidH I read some :P
20:32 echeveria buffalobill33: if you're asking that question, you clearly haven't evaluated the risks in handling other people's money.
20:32 buffalobill33 echeveria: can you clarify that please?
20:32 DSidH would core be able to handle say a million watch only addresses?
20:35 echeveria buffalobill33: you asked for an evaluation of cost for a system that handles other people's money, you should be able to answer that.
20:35 echeveria DSidH: it's not designed for that. many things in the wallet are inefficient.
20:39 DSidH echeveria: I see, so best would be to use bitcoind to get blocks and parse them myself
20:39 buffalobill33 echeveria: I don't know enough about the specifics of how a bitcoin node works to answer it correctly. I guess to rephrase the question: does it take 700 hours billable at $100/hour to write a btc payment processor which can send, receive and notify of incoming payments? The rest of the logic (coin selection, address tracking etc) will be written be a separate team.
20:40 DSidH Can someone help in understanding the test vectors at ? The private key hex seems to be much larger (38 bytes)
20:41 DSidH what encoding is the private key? Example: 79186670301299046436858412936420417076660923359050732094116068951337164773779
20:41 echeveria .. integer.
20:41 arubi it seems to be in decimals?
20:41 DSidH is it just int? But its much larger than 256 bits
20:42 DSidH also the op says "Private as hex "
20:42 arubi ;;calc log(77359564092138606367423909782286964438584967790833203478204963256314910737690)/log(2)
20:42 gribble 255.418114877
20:42 arubi ;;calc log(23984934282846201135004307705980641080962872643161302353166056218330425914143)/log(2)
20:42 gribble 253.728663698
20:43 arubi all fine :)
20:43 DSidH oh ok so he says hex but its not
20:43 echeveria buffalobill33: I was saying that's a problem in itself. if you're handling other people's money you need to be able to independently evaluate that.
20:44 DSidH thx. stupid thing, should have spotted myself.
20:45 DSidH all tests pass.. so vectors are good
20:45 DSidH these are some edge cases
20:45 arubi nice
21:02 arubi ohh I just noticed you're doing recoverable sigs DSidH
21:03 arubi DSidH,
21:05 DSidH arubi: which signatures are you referring to? In this post?
21:05 arubi there are usually 2 pubkeys and 4 addresses recoverable per signature, and sometimes it can be 4 pubkeys and 8 addresses, but only if the "signer" sets it on purpose :)
21:06 arubi yea
21:07 DSidH ok thats some other user but this is interesting fact
21:08 arubi ah I see you were just using the numbers to test your output
21:08 DSidH (s)he is not using deterministic ECDSA as the sigs dont match mine
21:08 DSidH yup
21:08 DSidH [02:35] <arubi> there are usually 2 pubkeys and 4 addresses recoverable per signature, and sometimes it can be 4 pubkeys and 8 addresses, but only if the "signer" sets it on purpose :)
21:08 DSidH please give some more references/details
21:09 DSidH key recovery is complex topic so I am not sure I'll get it
21:09 DSidH How do I know they are recoverable sigs?
21:09 arubi in core there's a feature called "signmessage"
21:10 arubi where you sign some text message and send the verifier your address and that message
21:10 DSidH yup but I could never find documentation of this
21:10 DSidH what encoding does it use?
21:10 arubi there are very good comments about it in libsecp256k1
21:11 arubi the encoding is just r and s as 32 bytes, and the recid byte encoded as in my paste, but it's not so simple
21:11 DSidH ok let me read. I assumed its simply DER encoded as in tx
21:11 arubi oh no, this one is much more fun :)
21:11 echeveria bitcoin doesn't really use DER anymore.
21:12 arubi it's all over the signatures
21:12 arubi err, transactions
21:13 arubi it doesn't use anything else in transactions.. yet. hopefully soon we'll just have 64 bytes constant length..
21:13 echeveria the format is far more restricted than DER
21:13 arubi oh yes, that's true.
21:13 echeveria DER allows unclear amounts of recursion, has dependancies on the underlying CPU integer sizes, all sorts of crazy you don't want.
21:14 arubi cool, in fact I don't want DER at all :)
21:14 DSidH arubi: does the "sign message" feature also use RFC6979?
21:14 arubi yep
21:15 echeveria I was more surprised to learn that there was more acceptable formats for public keys in bitcoin than 04,03,02.
21:15 arubi but the message hash is something you'll need to implement too. there is a header added, varints for lengths of the header and message..
21:15 arubi oh yea, the all elusive hybrid keys
21:15 echeveria there's supposedly one in testnet.
21:16 arubi would be cool if there's one on mainnet hiding behind p2pkh heh :)
21:38 DSidH arubi: can you please point me to the github link for the source code with comments for "sign message". I want to replicate those signatures (claimed to be from core)
22:00 arubi DSidH, for signmessage message format it's in bitcoin core repo and for recoverable signatures it's in the libsecp256k1 repo. I gotta go to sleep so can't search for it now, but probably tomorrow :)
22:01 arubi /night
22:01 DSidH no worries I found some links referring to magic bytes
22:01 DSidH I'll try those in the meantime
22:41 DSidH How do I use recoverable signatures? do I just send the signature and the address, using which the receiver can decide the right key?
23:49 spectra |Satoshi wanted us to increase the block size
23:50 spectra |many of the core developers sold out to Block Stream are the ones who vote against it
23:51 spectra |the lightning network will inevitably lead to centralized hubs with large liquity which whom we open accounts - banks