Transcript for #bitcoin-dev 2017/11/28

12:52 tomdickharry hi, i'm trying to find a wallet file on a computer, the file has no extension. im running windows and can't find a hex editor that can search a disk....anyone got any ideas? thanks
14:52 rgenito heyyyyy
14:52 rgenito how do you use SIGHASH_FORKID in the signature?
14:53 rgenito whilst broadcasting for Bitcoin Cash, I'm getting "16: mandatory-script-verify-flag-failed (Signature must use SIGHASH_FORKID). Code:-26"
14:53 rgenito been googling around. any tips will be great! I'm doing all of this straight from the command line
14:59 wumpus what did you use to sign?
15:00 wumpus there are hacks floating around to patch bitcoin core to sign transactions for bch, for example: https://github.com/laanwj/bitcoin/commit/98f79a7243bbc5eccee2649ef19178c1908dc555 then use signrawtransaction with "ALL|ABC"
15:14 rgenito wumpus: i used the bitcoin cash version of bitcoin unlimited
15:15 rgenito wumpus: https://www.bitcoinunlimited.info/download
15:15 rgenito and thank you for the response! :)
15:19 wumpus ok can't help you with that
15:20 wumpus apparently your client signed the transaction for bitcoin instead of bcash
15:21 rgenito wumpus: ah ok, ya.... i'll agree.
15:22 rgenito wumpus: what bitcoin cash node do you recommend then for manual transactions? apparently bitcoin unlimited isn't the answer
15:30 rgenito wumpus: ok i got bitcoinabc :)
15:31 rgenito *sigh*, signing a transaction is now "missing Amount"
17:04 cluelessperson Can someone help me understand SegWit from the ground up?
17:31 mlz cluelessperson, start with this video: https://www.youtube.com/watch?v=zchzn7aPQjI&feature=youtu.be&t=101
17:34 cluelessperson mlz: thanks
17:35 cluelessperson mlz: I've observed that SegWit moves signature information "witness data" into a "SegWit field" in the block
17:35 cluelessperson also, I'm confused by something
17:35 cluelessperson Are blocks effectively a larger size in bytes as a result of segwit?
17:36 cluelessperson segwit has an "effective weight" of 4 million weight units
17:38 arubi it's not true that segwit moves script and signature data to a different place in a block. it does move it to a different place within a transaction, and also commits to all the witness data as an output in the generation transaction
17:39 arubi yes for the second question. it's valid now to have a block that's almost 4 megabytes and made up of almost 4 million witness bytes
17:39 arubi it's "almost" because the non witness data would just be the generation transaction itself
17:48 cluelessperson arubi: yeah, that's what I meant, sorry.
17:48 cluelessperson arubi: What's a generation transaction?
17:48 arubi what's usually called coinbase transaction
17:48 cluelessperson ah, got it
17:48 arubi it's not a good name because "coinbase" is a term in itself
17:49 arubi same ambiguity with the term "scriptsig" :(
17:49 cluelessperson arubi: I'm confused because some people have almost demanded a blocksize increase, and I believe I saw it reasoned that even 2MB is so much, I don't understand why SegWit enables 4 megabytes
17:49 cluelessperson isn't that still too much?
17:50 arubi it's a lot yea, I agree
17:51 cluelessperson arubi: I feel I've been misled or that I'm still confused.
17:51 arubi how so?
17:54 mlz arubi, miners can mine their block rewards to a Segwit address, correct?
17:55 arubi sure
18:57 cluelessperson arubi: Because, we've specifically fought against a block size increase.
19:01 arubi well, segwit fixes a lot of issues with just a blunt increase. this increase is done in a safe way
19:02 cluelessperson arubi: What does that mean? I'm all for fixing transaction malleuability, but I'm concerned with the current transaction system still in place.
19:02 cluelessperson people aren't using segwit yet
19:03 arubi a block can't have more than 1 megabyte of non witness bytes
19:03 arubi it's not the size itself that's the issue, it's validation
19:03 arubi and segwit
19:03 arubi and segwit's bip143 fixes a lot of validation slowdowns
19:06 arubi transaction malleability is fixed just by moving the signature data to the witness area in a transaction. segwit the software made specific scripts' executed checksig operation to behave in a safer manner when hashing the transaction data
19:08 cluelessperson arubi: I don't understand.
19:10 arubi it's a lot less resource intensive to validate checksigs in segwit scripts than it is in pre-segwit scripts like p2pkh. basically pre-segwit checksigs are hashing the transaction in a way where each checksig hashes entirely different data
19:11 arubi you have to set up the "sighash" from scratch for each input where a checksig is made, so many inputs and many outputs in a single transaction can get to a very large amount of hashed data
19:12 arubi with segwit, you set up the data to be signed in such a way that the vast majority can be re-used by other checksigs (or verifies)
19:15 arubi the difference is, say you have an unsigned transaction with 10 inputs, and its size is 1kb, then a verifier would be hashing 10kb of data to check it. 1kb for each checksig in each input
19:16 arubi so if the consensus maximum is 20,000 checksigs in a block, and a block size (pre-segwit) is 1 megabyte, then you can guess what the worst case is :)
19:16 arubi well, it's a bit less than 1mb*20000, but not by a lot hehe
19:21 alt0id hey
19:21 alt0id any dev's here?
19:21 alt0id devs*
19:22 alt0id why does my btc client sometimes stop downloading data when syncing?
19:24 eck I have a question about the p2p protocol... for 'addr' messages, am I supposed to use the current timestamp, or the timestamp I last talked to the node I'm advertising?
19:26 Chris_Stewart_5 alt0id: I'm guessing, but it might be to verify that the blocks you received are correct
19:26 Chris_Stewart_5 alt0id: I think you could attack a node by sending a BUNCH of data at once, and then let them verify it after the fact
19:27 Chris_Stewart_5 if you sync x blocks, verify those x blocks, sync x block again, verify x blocks again
19:27 Chris_Stewart_5 you don't have to worry about that attack vector. Again i am guessing though
19:28 Chris_Stewart_5 eck: https://bitcoin.org/en/developer-reference#addr
19:28 Chris_Stewart_5 "Nodes advertising IP addresses they’ve connected to set this to the last time they connected to that node"
19:28 eck thanks
19:28 eck I was looking at the wiki protocol docs, I should use this instead
19:28 Chris_Stewart_5 np
19:58 alt0id Chris_Stewart_5: so closing/re-connecting is a bad idea?
20:23 cluelessperson alt0id: who knows, reorgnization, compression, etc?
21:13 cluelessperson Can someone help me understand how to communicate with nodes via python?
21:13 eck for the p2p protocol?
21:13 eck or the json rpc protocol?
21:14 eck this is the de facto standard for python https://github.com/jgarzik/python-bitcoinrpc
21:14 cluelessperson eck: the p2p
21:15 eck https://github.com/petertodd/python-bitcoinlib
21:15 cluelessperson eck: Aren't those both for the rpc?
21:15 eck bitcoinlib can parse the p2p network messages
21:16 cluelessperson eck: I want to learn how to parse them myself.
21:16 eck i am writing a p2p client in c++ right now, the protocol is pretty simple if you want to try to write your own
21:16 cluelessperson eck: Yes, that's whta I'm looking for
21:18 eck this is what i have so far for the encoding/decoding logic https://github.com/eklitzke/spv/blob/master/src/message.cc
21:18 eck https://bitcoin.org/en/developer-reference describes all the messages
21:19 eck it would be a lot less code in python due to how much easier reflection is in python
21:20 cluelessperson eck: I don't know ++ :/
21:21 eck tl;dr is you do some dns lookups to some seeds, use those to bootstrap your peer list
21:21 eck and then it's a pretty simple framed protocol as described in the bitcoin.org docs
21:21 cluelessperson eck: how do I connect to and query a node though?
21:21 cluelessperson say I hopen a socket
21:21 eck you bootstrap the list using dns, then later you get peer announcements from your peers. the peers all listen on a well known port.
21:22 cluelessperson eck: I'm having trouble with the communication part
21:22 cluelessperson like, how do they wrap messages?
21:22 eck it's a framed protocol over tcp
21:22 cluelessperson what protocol?
21:22 eck the message frame has a magic constant, the message type, the payload length, and a checksum
21:22 cluelessperson ah
21:22 cluelessperson that's all?
21:22 eck custom framed protocol on top of tcp
21:22 eck well after that there's a payload
21:23 eck which you need to decode as well
21:23 cluelessperson decode? you mean deserialize?
21:23 eck you can skip the message types you don't understand though, since the message header has the payload size
21:26 eck the protocol is kind of stateful, so if you want to try to write your own client i recommend using an event loop (e.g. tornado for python) to handle timers and whatnot
21:33 cluelessperson eck: I'm not at all familmiar with tornado
21:33 cluelessperson eck: what do you mean stateful?
21:56 eck some of the message types are request/response where you need to remember your request to validate the response, and some things are supposed to be expired from memory after different amounts of time