Transcript for #bitcoin-dev 2017/10/01

03:31 wallet42 esotericnonsense: that's what i though it would do but it felt highly inefficient. but i guess that's the whole point why it's deprecated
03:33 esotericnonsense wallet42: i'm not fully up to scratch on the chainstate format (past or present); one thing I remember seeing is that it should be possible to prune spent outputs from it while keeping other outputs within the same transaction
03:33 wallet42 yes the new format is actually quite straigtforward
03:33 esotericnonsense not sure if that already happens; if it does then getrawtransaction could only work for transactions that have all outputs unspent
03:34 wallet42 yes it needs at least 1 utxo
03:34 wallet42 getting the block height
03:34 wallet42 then looking up in blockindex
03:34 esotericnonsense ah that's it. right. so it requires 1 utxo _and_ unpruned
03:35 wallet42 yep
03:35 esotericnonsense (thinking about it on a theoretical level rather than how it's actually implemented, i don't know if getrawtransaction works at all on a pruned node)
03:37 wallet42 it does, but only best effort
03:38 wallet42 pruning is also just deleting whole blk files
03:38 wallet42 the blockindex has pointers to those files
03:38 wallet42 the pointers dont get removed
03:38 wallet42 but if the data at pointer doesnt exist anymore, it fails
03:44 esotericnonsense of course :P
03:44 esotericnonsense okay
18:20 hidden so quiet
19:03 djdjjj question re. TXs: What's the reason for the actual redeemer's pubkey being included in the tx, given that it already can be found by the txin/index info? (I'm going through https://en.bitcoin.it/wiki/Transaction)
19:15 arubi djdjjj, you mean at signing time? you're right. it doesn't have to be there at all. the prevtx's txid explicitly passes that information to the signature anyway
19:15 djdjjj arubi: signing time, yes. So is there any particular reason it's there?
19:16 arubi djdjjj, maybe if you could include input scripts into sighash, but other than that it's really not needed. security stays the same
19:18 djdjjj thx arubi. Another one: when nodes verify if a tx is valid by validating the chain of ownership, how far back (in number of txs) do they go?
19:19 djdjjj (I'm a newbie to bitcoin btw)
19:20 arubi they just need to look at their known database of spendable outputs
19:20 arubi if it isn't there, it's invalid. full nodes have the most updated database at all times if they're connected to the network
19:22 djdjjj So if I send 1BTC to Bob, full nodes have to validate that I'm the rightful owner of my referenced inputs, right?
19:23 arubi that's done by verifying you signatures against the inputs' scripts, right
19:23 arubi (brb)
19:23 djdjjj right. But don't they have to also verify the inputs of my referenced inputs and so on?
19:24 djdjjj eg. what if the referenced input of one of my inputs is invalid
19:24 arubi no, because you only spend the previous tx's outputs, not the whole chain
19:24 arubi then the input itself can't exist in the utxo set
19:24 arubi which is the database of spendable outputs
19:24 djdjjj come to think of it, this is probably not needed since this validation work of those previous inputs is already done since it's already included in a block
19:24 arubi :)
19:25 djdjjj oh so this utxo set is a global one? Every node knows all the UTXOs on the chain?
19:26 arubi yes, it's what all full nodes agree on
19:27 djdjjj makes sense now, thanks :)
19:27 arubi yw
19:27 djdjjj so there's that invariant: a UTXO's txinputs cannot be UTXO themselves
19:29 arubi oh sure, because they haven't been created yet when you try to spend them
19:30 djdjjj otherwise that would be double-spending, wouldn't it?
19:31 arubi not saying it can't happen, but you'd need to break sha256 to set it up
19:31 arubi you're trying to set up a tx that when redeeming an input has the same txid as that input. sounds possible, but not with current tech :)
19:32 djdjjj ah right
19:34 djdjjj I didn't knew that UTXO database is part of the consensus protocol btw. I thought only the proof-of-work chain was part of it
19:35 arubi right the utxo is the grand thing everyone agrees on because everybody follows the same consensus rules
19:36 djdjjj I'm confused by these answers however that state that each node computes the UTXO set on its own https://bitcoin.stackexchange.com/questions/49393/is-the-utxo-set-computed-from-scratch-by-new-fu...
19:37 arubi each node build their own view of what the utxo set is
19:37 arubi but since everybody run the same rules, they will eventually agree
19:37 arubi they might disagree sometimes, but it converges very quickly
19:38 djdjjj ok I get it. So by looking at the UTXO set you can effectively compute how much BTC exists right now?
19:38 arubi yep
19:39 arubi there's a specific call in bitcoin core, 'gettxoutsetinfo' that does that
19:47 djdjjj re. my initial question, someone replied with a slightly different answer than yours I think: https://www.reddit.com/r/BitcoinBeginners/comments/73nmjr/technical_question_why_is_the_redeemers...
19:47 arubi which answer do you mean?
19:48 arubi the address isn't playing a part in anything. addresses are not part of the actual protocol. the previous txid:index points to a scriptpubkey
19:48 arubi in case it's p2pkh (send to address), it's obvious what should be the script spending it
19:49 arubi at signing time, that scriptpubkey is signed, but it's redundant. it was already hashed as part of the input's txid itself
19:50 arubi it already exist in sighash, there's no need to explicitly put it in there again, but now I'm remembering op_codeseparator
19:50 djdjjj my mistake then, I was talking about validation time, not signing time.
19:50 arubi so actually djdjjj if we didn't have op_codeseparator, I'd be right, but we do and for some special cases, you do need the script signed too
19:51 arubi validation is the same. you know the utxo, you know the scriptpubkey required to spend it
19:52 djdjjj I guess my confusion comes from the fact that I thought that addresses were just the public keys
19:53 arubi addresses are just strings that when you tell a wallet to "send to them", it actually sends to a standard scriptpubkey that has the public key's hash instead of the public key itself
19:53 arubi to redeem it, in the spending transaction, you provide the public key itself, and a valid signature. so the previous tx's scriptpubkey is executed with these two as inputs
19:54 djdjjj so address is a string representation of the hashed value of a public key?
19:54 arubi right, and some other non interesting things. mainly the pubkey's hash
19:55 djdjjj so only the redeemer knows his/her own public key?
19:55 arubi yes
19:56 djdjjj the other peers only know his/her hashed keys (ie. addresses)
19:56 arubi right, and to prove ownership of that key, the owner has to sign the hash of the key
19:56 arubi which is just as hard as signing the key itself
19:56 arubi (or easy if you have the private key)
19:57 arubi that's why I initially said you could just use the previous txid, because it already contains the hash of the pubkey, but special singing conditions require the actual executed script too
19:57 djdjjj so p2pkh is the simpler format AFAICT, and the most common one
19:58 arubi it's most common yes, hopefully not for long as p2wpkh replaces it :)
19:58 arubi (same idea, just a more efficient script)
20:02 djdjjj it's clear a few paragraphs below in the wiki: "A Bitcoin address is only a hash, so the sender can't provide a full public key in scriptPubKey. When redeeming coins that have been sent to a Bitcoin address, the recipient provides both the signature and the public key. The script verifies that the provided public key does hash to the hash in scriptPubKey, and then it also checks the signature against the public key."
20:02 djdjjj it all makes sense now :)
20:02 djdjjj thanks a lot
20:03 arubi the paragraph is lacking really
20:03 djdjjj x_x
20:03 djdjjj in which way?
20:03 arubi oh sorry, it does
20:03 arubi "also checks the signature against the public key"
20:03 arubi you're welcome :)
20:04 arubi I missed the last "and then.." at first
20:06 djdjjj so the actual public keys are not visible in wallets users (they only see addresses) but are visible in full nodes
20:06 arubi address owners can run 'ismine <address>' and see the pubkey, but usually well done wallets try to avoid letting the user handle any keys at all
20:07 djdjjj and the public keys in TXs appear only in the "scriptSig" part (talking about p2pkh addresses)
20:07 Sentineo how did a p2pk addresslook like? was it the pubkey in base56? Or just plain?
20:08 Sentineo djdjjj: yes, only in scriptSig (or as I like the more descriptive name unlocking script)
20:08 arubi djdjjj, right, not a part of the recorded transaction at all
20:09 arubi Sentineo, there isn't an address for that. it's just a pubkey and checksig after
20:09 arubi so sending to a pubkey would just be sending to a script "<pubkey> checksig"
20:09 Sentineo ok, so historicaly base58 was cr3ated for p2pkh?
20:09 arubi not base58 either, just hex pubkeys :)
20:10 Sentineo are those still valid?
20:10 arubi sure, and standard
20:10 Sentineo need to try on regt3st ;)
20:10 arubi multisigs too
20:10 arubi as bare scripts that is, not p2sh wrapped
20:11 Sentineo yeah, remember those
20:11 Sentineo been playing with them
20:11 arubi miners used to pay themselves with a bare 1-of-1 multisig, weird, maybe to make the software count -20 sigops and have some safety margin for sigop count
20:11 djdjjj so bare public keys are only used to verify the signature and to create addresses?
20:11 arubi I don't know the reason. not sure if minres still do :)
20:12 arubi and multisig scripts
20:12 arubi well these are wrapped in a hash themselves, so they're not bare, but you do interact with keys for that
20:13 djdjjj I see
20:13 djdjjj so `scriptPubKey` is a little misleading name..
20:13 arubi scriptsig too
20:14 djdjjj well that's kinda vague too yes
20:14 Sentineo yep
20:14 Sentineo well because it was mostly cod3
20:14 Sentineo code
20:14 arubi it's all because when these were made up, they had a bit different uses
20:14 Sentineo yeah
20:14 djdjjj as a sender, `scriptSig` contains my public key and my signature, and `scriptPubKey` contains the recipients hashed public key (ie. address)
20:14 arubi scriptsig, you used to be able to execute scripts on there. you still can to some extent, but not very useful
20:15 arubi scriptsig is the script that runs before the input's scriptpubkey us run
20:15 Sentineo djdjjj: scriptSig js the unlocking script and scriptPubkey, the locking
20:15 arubi right now, scriptsig is only confined to pushes of data, so the script just pushes stuff like pubkeys and sigs to the stack
20:15 arubi and when the scriptpubkey runs, these are its inputs
20:16 Sentineo djdjjj: so when you send something you lock it (givr the crypto puzzle)
20:16 arubi so for a p2pkh, the inputs on the stack, put there by scriptsig would be the pubkey and signature
20:17 arubi for a p2pk, it's only the signature (pubkey not recorded, but in sighash as previous scriptpubkey), for p2pkh it's both (hash not recorded, but in sighash as previous scriptpbukey)
20:18 Sentineo djdjjj: and to spend it, you have to unlock it by providing the scriptSig.
20:19 Sentineo djdjjj: read this and it will make more sense ... https://github.com/bitcoinbook/bitcoinbook/blob/second_edition/ch06.asciidoc
20:20 arubi no! read http://enetium.com/resources/Bitcoin.pdf !
20:20 djdjjj question: the `pubKey` in `scriptSig` is *my* public key, right?
20:21 djdjjj (not the recipients)
20:21 arubi yes, yours
20:21 arubi the one redeeming
20:22 arubi the recipient's is unknown yet :)
20:22 arubi (sending to address, right)
20:22 djdjjj I'm kinda confused by this: https://en.bitcoin.it/wiki/Transaction#Pay-to-PubkeyHash
20:23 arubi specifically?
20:23 djdjjj when the OP_EQUALVERIFY comes into play, it says that it compares `pubHashA` with `pubKeyHash`
20:23 arubi yes, and fails the script if it isn't equal
20:23 djdjjj why would those be equal? Isn't `pubHashA` _my_ hashed public key?
20:23 djdjjj and `pubKeyHash` the address of the recipient?
20:23 arubi yes, it's compared to the hash in the input's scriptpubkey txid:index
20:24 arubi no no, that pubkeyhash is in the outputs
20:24 arubi things in scriptsig are input stuff, either scripts at redemption or previous scriptpubkey at signing time
20:25 arubi look at page 29 in the pdf I linked
20:25 djdjjj ah sorry I got it
20:25 arubi oh alright
20:25 djdjjj this is the whole process that checks that I'm the rightful owner
20:25 Sentineo djdjjj: you have to concatenate scriptSig and scriptPubkey and lookmat it as one script
20:25 djdjjj if the coins I'm trying to spend
20:25 Sentineo djdjjj: if it returns true you transer bitckin ownership
20:26 arubi ownership is checked by making you sign both the previous txid:index and the script being executed
20:26 djdjjj I mean, that whole "checking process" described in that page refers only to "checking that I'm the owner of the UTXO's I'm trying to spend"
20:26 arubi yes, that's what validation of transactions does basically
20:26 djdjjj I thought that it was validating the whole transaction somehow (meaning involving the sender too)..
20:26 djdjjj yes yes
20:28 djdjjj so that `pubKeyHash` there actually refers to the `pubKeyHash` of a _UTXO_ that I'm redeeming, not the `pubKeyHash` that I'll send the coins to
20:29 arubi yes that's right
20:29 djdjjj that PDF is awesome btw
20:29 arubi usually signers sign all the outputs in a transaction, but they always only sign their own input
20:30 arubi they do sign other txid:index inputs though, just not scriptpubkeys
20:30 arubi or sequences, you'll see :)
20:54 djdjjj last one: which data are signed in the scriptSig signature part?
20:55 djdjjj I guess it's a hash of the tx inputs themselves?
20:56 djdjjj or the tx-to-be-created?
20:58 arubi it's everything, prevtxs and current outputs
20:59 arubi anything that isn't signed is free to change in flight by anyone, so it's pretty thorough if you don't use very exotic setups