Transcript for #bitcoin-dev 2017/12/04

02:13 eck when doing the initial sync of block headers, am i supposed to query multiple peers for the same block hash to detect forks?
15:10 denisx 2017-12-04 13:27:08 connect() to [::84.251.161.205]:8333 failed: Network is unreachable (51)
15:10 denisx had this in my logfile, is this a bug?
15:10 denisx what is [::84.251.161.205]?
15:14 mlz denisx, it's an iP
15:15 mlz your node tried to connect to that IP and failed, nothing unusual and not a bug
15:15 denisx mlz: doesnt look like one, it is not ipv6 and it is not ipv4
15:16 mlz denisx, what do you think it is?
15:16 denisx a bug ;)
15:17 denisx ok, I looked it up, it is an ipv4 address mapped in ipv6
15:17 denisx not a bug ;)
15:19 denisx it is calld IPv6/IPv4 Address Embedding
15:19 denisx http://www.tcpipguide.com/free/t_IPv6IPv4AddressEmbedding.htm
17:27 BlueMatt it'd be nice to move #11556 along....more than a month for a string change is...ugh
17:27 BlueMatt more review would help :)
17:49 gevs Hello people! I have been fighting to recover some USDT from a copay multisig and would like some insights from bitcoin network experts :) I am a total noob with Bitcoin, I have basic blockchain knowledge due to professional practice but am fighting with this topic because it includes: Bitcoin, OP_RETURN, Omni USDT & Copay multisig accounts. I have learned and read a lot about the subject, here is a link to the bitcoin stackexchange question I have
17:49 gevs posted which also links to more details on bitcointalk (quite a complicated topic to describe..). Any help is much appreciated, here is the link: https://bitcoin.stackexchange.com/questions/64105/bitcoin-protocol-op-return-omni-usdt-tether-cop... Thank you!
17:59 arubi so much links to follow, just show the unsigned transaction already :P
18:01 arubi "mandatory-script-verify-flag-failed" <- means that the script is not failing in the multisig part
18:02 arubi unless, what version of core are you running?
18:05 gevs oh hey :) that error i passed now :P but yes it was already giving me insurance about the mutlisig part
18:06 gevs now the error is "64: scriptpubkey"
18:06 arubi what's the signed tx hex then?
18:07 gevs 0100000002515b4e1ebff9fb500ecb1d94b184042b9d3d580829eee288b55fa4006534419101000000b500483045022100a4c41919ca7b9f5371f6dc90b8064defc03c89b6c938f04928cedcb60b1d6f4002207620c409448bfe88d78fb0ecb970c3986e01325b39b68460a94406c8f6b55797014c6951210247bcd2c9e1bf8bfc9e567c8f9c81ef5f2c5356e544ab7b9141b7ef93f272da9f2102771dbfd6b9e6183e893c07ea49cfc781d1836e42b6425ff2a59100cc2e9b11222102ab0456bde3a2a8f5066fa4827c8a00c9388e6da1fdc22f90c406d89112c8bacb53aeffffffffcb
18:07 gevs 0e9efba54a6acee3d8b09df2b9527ca05fb1c61b6f893ad5c8ee2cc4d6f54e00000000b500483045022100d7845a9e0e854d675451863027eefaa7b01d5ae87ee700c3b40012e2c94a30760220053e223dca51f234887226871cbbe94678913ab8b96c60edff8e454a2efc4876014c6951210247bcd2c9e1bf8bfc9e567c8f9c81ef5f2c5356e544ab7b9141b7ef93f272da9f2102771dbfd6b9e6183e893c07ea49cfc781d1836e42b6425ff2a59100cc2e9b11222102ab0456bde3a2a8f5066fa4827c8a00c9388e6da1fdc22f90c406d89112c8bacb53aeffffffff03000000000000
18:07 gevs 0000176a146f6d6e69000000000000001f000000002faf080087c4090000000000001976a914f40da014c137ad88006608834887556944e2dfeb88acf0490200000000001976a9142168fe52d8c1cbc656a544859c4193dd4b92e23788ac00000000
18:08 arubi alright let's see
18:08 gevs great thanks
18:08 gevs so let me explain real quick - if you import it through any API - you will see currently there is 3 outputs
18:09 gevs thats because i was unsure about how OMNI tells the "destination" - it seems "new protocol" uses the *first VIN* - while the old protocol used to *tell the largest input by sum* to know the destination
18:10 gevs the first output is the OP_RETURN with omni payload, the second output is the one im talking about 0.00002500 (dust BTC for omni destination) - the third output is a BTC output with which i make sure that at least 0.00050000 is used as the fee (because in the input linked, there is ~0.002)
18:11 gevs second output can be skipped, without it i also get a 64:scriptpubkey error :) greg over :P
18:16 arubi the redeemscript doesn't match the addresses
18:16 arubi so it fails at the p2sh stage, way before the multisig is even executed, so maybe hold off on setting up the outputs for now :)
18:17 arubi you got the redeemscript for hash D7FB889271798A9244F47320AEC28B926A2BB7A4, but you want one with hash 666E5E7041CD7A9DF7075D2F3FC79B6D53DF7963
18:17 Sentineo does not sound good :)
18:18 arubi maybe just wrong ordering of the pubkeys :)
18:18 gevs ha
18:19 gevs i was sure it was this 666 hash at the beginning
18:19 gevs but then my hd-from-xpubs tool gave me exactly that redeemScript and i thought i must be on the right way
18:19 gevs ordering of keys i tried a lot of different orders (lexicographically ordered as mentionen in the scriptPubKey preferred)
18:19 gevs also tried with 2 and 3 keys but cant come on that 666 hash
18:20 gevs when you say it fails at the p2sh stage - maybe im building my outpoint incorrectly?
18:20 arubi no, you have the wrong redeemscript
18:20 arubi the "HASH160 <scripthash> EQUAL" fails
18:20 arubi it's... not EQUAL :) false is left on stack, and you script evaluates to false
18:21 gevs ok it fails because i have d7fb instead of 666 in my produced hash
18:23 arubi the condition to spend the input is to first provide a redeemscript that hashes to 666..., then when executed leaves a value "true" on the stack
18:24 arubi so if that's some copay multisig, you need to find the pubkeys that were used, then trying out some 1-of-n or 2-of-n is simple
18:26 gevs yes i have those keys
18:26 gevs with certainty because i can also reproduce the right XPUB with the said derivation path
18:26 gevs (the XPUBs displayed for the 3 cosigners, in the copay app wallet information page)
18:27 arubi well, maybe the script also has some timelocks set in it
18:27 arubi like CLTV or CSV. you need the complete redeemscript
18:27 gevs wouldnt that appear when you check the input of the transaction : 9141346500a45fb588e2ee2908583d9d2b0484b1941dcb0e50fbf9bf1e4e5b51
18:28 arubi no, it's in the redeemscript :)
18:28 gevs ah ok its in the 666
18:28 gevs got it
18:28 arubi that too, in some metaphysical sense
18:28 arubi it's encoded in the redeemscript. the 666... part is its hash
18:28 gevs yep of course
18:29 gevs how can i know about that :')
18:29 gevs copay does not mention something like this
18:29 Sentineo well you need to know the redeemscript :)
18:30 gevs and from the code i have seen in their *minified* (brr) copay recovery tool, they just use the same derivation path as me
18:30 gevs m/44'/0
18:30 Sentineo to claim it, but not sure what you guys are doing, it is omni for me :D
18:30 gevs oups *m/44'/0'/0'/
18:32 gevs Sentineo, im trying to extract usdt from a copay multisig
18:32 arubi gevs, if you have access to the same wallet with these xpubs in it, you can try to send coins back to yourself and look at that redeemscriot
18:32 Sentineo yeah that I understood, but never played with that omni thing
18:32 arubi it'll show the structure at least, and you could just replace the pubkeys
18:32 gevs so it is omni yes - browsing some repositories i found how the omni payload is created (in several parts)
18:32 gevs ah ok :)
18:33 Sentineo wow arubi that is a really neat one!
18:33 gevs wait what - with what client?
18:33 arubi if only these services would offer testnet :) gevs the one normally used to spend those copay funds?
18:33 gevs i have access to the 3 cosigners
18:34 arubi do you have examples of past transactions?
18:34 arubi (brb)
18:34 Sentineo what he is suggesting you do a similar tx as you are trying to craft
18:34 gevs ha, im guessing its something like an exchange ^^
18:34 Sentineo if you show him a similar tx (as this one) he can help you with the redeemscript format
18:34 gevs the Omniwallet probably wont let people send money on a copay wallet - yet unsure here too
18:35 gevs arubi, i dont have any other previous transaction - i did this 8 USDT transaction myself
18:35 gevs i reproduced what my customer asked me :)
18:36 gevs he sent tether on a copay multisig - then i reproduced this by sending 8 USDT myself - on a copay multisig with a little btc for the fee - which then locks those USDT in the copay multisig
18:38 gevs ok but i totally get the principle
18:39 arubi welp, it's either some timelock additions to the script, one of the keys is wrong, missing, or encoded differently (uncompressed?)
18:39 gevs the money was sent from changelly so sadly i dont have the client code here :/
18:39 gevs oh hell i was sure i would hurt on compressed/uncompressed after all the time i read about it haha
18:40 gevs will do some more testing about the keys as it seems its where the problem originates
18:40 arubi I can't imagine anybody uses uncompressed keys now. it's probably that their service uses timelocks so in case they disappear, you can still redeem the funds
18:41 gevs ok - i will need to read about timelocks too, havent had that one on my documents yet
18:41 arubi gevs, did you see this? https://github.com/bitpay/copay/issues/6329
18:41 arubi taking back my comment about not using uncompressed stuff :)
18:43 gevs thanks for the link checking this
18:47 gevs i havent seen this but i have learned it from the code :D
18:48 gevs in copay the bip32 XPUB is derived at m/44'/0'/0' and cosigners are then xpub/0/1 xpub/0/2 and xpub/0/3
18:49 gevs but i didnt get the redeem scripts reading method, great checking out :)
18:50 arubi <gevs> in copay the bip32 XPUB is derived at m/44'/0'/0'...
18:50 gevs i see following ASM: 30450221008dad53ca84070e9328f7a1a33eecd490258958f10d1a1dccf5709af8cad1339002205d45b357727021f57075b4db9f309d266dd0ee90112b6df6e11a6a162a27efa001 04fcf07bb1222f7925f2b7cc15183a40443c578e62ea17100aa3b44ba66905c95d4980aec4cd2f6eb426d1b1ec45d76724f26901099416b9265b76ba67c8b0b73d
18:50 arubi makes no sense. this is a hardened derivation notation
18:50 arubi or, they mean m/44'/0'/0'/... ?
18:50 gevs ok enlighten me because i really thought im ok with key derivation now
18:51 gevs yes
18:51 arubi that's an uncompressed key, where is that from?
18:51 gevs xpub/ is basically an alias for "m/44'/0'/0'"
18:51 arubi the last "/" is important
18:51 arubi (for the previous line) I see what you mean now :)
18:52 arubi where's the signature from?
18:52 gevs this uncompressed ASM comes from the inputs of the transaction: https://api.smartbit.com.au/v1/blockchain/tx/9141346500a45fb588e2ee2908583d9d2b0484b1941dcb0e50fb...
18:52 gevs this transaction's outputs is where the USDT to 3B2d4... are
18:52 arubi but that's just some address paying. that's the previous output being redeemed
18:52 gevs and with omniwallet.com or .org im able to see those 8 USDT then on the address with all corresponding properties
18:52 arubi it's unrelated to what you're trying to spend, unless you used that key in there
18:53 gevs ah ok gotcha
18:54 gevs i can send one transaction with copay now
18:54 gevs just sending a few btc
18:54 gevs this will help right?
18:56 arubi yep
18:59 gevs https://blockchain.info/tx/517a0ad4bf4cc423ce578f043a13e98d405902c08b1bfcac92b199ba3fd2cc39?show_...
19:00 gevs ok so what you are saying - now i should try to rebuild this transaction i just made ? because i kind of know the content?
19:02 arubi gevs, that is the exact redeemscript : 522102109c754588a5e6de512a68b0e82fa8ed6520f5fd57f6b41315e3d3c2b8f92ac2210318e0167d697e9366f17c2e83ac21c7e66ecc9427884d083887c8d3b6aa3857d8210350bc7cb1f4632c42ad092b1b825beea81ad4a663fa1c365c95d09ff44a11f30353ae
19:03 arubi it's a 2-of-3 with pubkeys 02109c754588a5e6de512a68b0e82fa8ed6520f5fd57f6b41315e3d3c2b8f92ac2 0318e0167d697e9366f17c2e83ac21c7e66ecc9427884d083887c8d3b6aa3857d8 0350bc7cb1f4632c42ad092b1b825beea81ad4a663fa1c365c95d09ff44a11f303
19:03 arubi all are different from what you had in mind :)
19:05 gevs oh no dont worry i tried the 2 of 3 first :D
19:06 gevs because thats what copay says on the app too :P
19:06 arubi not what I mean. your redeemscript uses 0247bcd2c9e1bf8bfc9e567c8f9c81ef5f2c5356e544ab7b9141b7ef93f272da9f 02771dbfd6b9e6183e893c07ea49cfc781d1836e42b6425ff2a59100cc2e9b1122 02ab0456bde3a2a8f5066fa4827c8a00c9388e6da1fdc22f90c406d89112c8bacb
19:07 arubi which are three different keys
19:11 gevs but yep exactly
19:11 arubi so, you can derive them in your software?
19:11 gevs yes :)
19:11 arubi sweet :)
19:11 gevs thats what gave me the first insurance about the multisig part
19:12 arubi so what was the error? what made it derive the wrong keys at first?
19:12 gevs wait wait
19:12 gevs that is the set of keys that i have been using since the beginning
19:13 gevs i havent tried to reproduce that transaction with my code now.
19:13 gevs my commands should derive exactly those keys, and order them lexicographically as its done in your message
19:13 arubi are you not seeing that these are two different sets of keys?
19:14 arubi the signed transaction that you provided uses the wrong set. the set you need to use is the one from the 517a0ad.. tx
19:22 gevs oh man im sorry
19:22 arubi no worries :)
19:22 gevs i didnt read the second message with "not what I meant" - and when i referred back to the keys, i saw those, which are the three im using for my cosigning
19:22 gevs damn ok- great :)
19:23 gevs will check a few details and come back if i have issues after finding those keys!
19:24 arubi alright
19:31 gevs thank you a lot i was somehow sure about the keys im using :/ because out of the xpubs from copay, it would show me that same list of public keys, for the redeem script i have been producing.
19:32 gevs its hard to explain, lots of inputs with different paths, different xpub/xpriv to derive, etc.. hope it'll get me a little further, I do like the fact that it seems to be a key problem because i should be able to solve it.
19:32 arubi I was guessing it's some kind of path issue
19:33 arubi gevs, I'm guessing the exported pubkeys copay gives you are at the edge of the hardened path
19:33 arubi so probably the last 0'
19:34 arubi then you start deriving public keys from that point on. is that what you're doing?
19:35 arubi and then if the xpub is at 0', then 0'/0/1 is the first master xpub for signer 1, 0'/0/2 master xpub for signer 2, and so on
19:36 arubi and you treat these master xpubs as the "new" root for whatever path for a specific set
19:40 gevs 1) i do a bip39 seed derivation of the mnemonic - based on the hexadecimal representation of this i create a bip32 HD key
19:41 gevs 2) out of this hd key i derive the path m/44'/0'/0' which is the XPUB path (keep in mind i do this for each cosigner, with each mnemonic)
19:41 gevs ha
19:42 gevs 3) is where its wrong i think :D im not deriving further xpub/0/0 for example which would make an absolute derivation path of m/44'/0'/0'/0/0
19:42 arubi indeed :)
19:42 gevs i see this just now so i might get back to the code thank you for the support here
19:42 arubi cheers, let us know. little docs with copay and this is valuable info
19:45 gevs yep will do and the code is on github for people who are ok with laravel for CLI :P it was the fastest i could think of as i use it daily :)
19:49 arubi gevs, you don't have to tell me. I have these things written in gnu bc and bash
19:50 gevs haha xD im glad i passed the paranoia step and kind of know which data im allowed to post in here and stuff :P
19:51 arubi hehe, kinda tempted to open a mock copay wallet just to figure out the paths, but I need to have dinner first :)
19:54 gevs im pretty sure about my paths haha - with the multiple commands i have written i am able to check starting from XPUB which address it will give, and also which redeemScript and scriptPubLKey - and then with the wallet:derive command i can get the private keys for the said paths
19:55 gevs kind of made me learn *a whole bunch* haha because i come from a less cryptographic blockchain :P im not here to advertize easiness of integration or whatever, i knew what i was getting into and im overly glad to learn so much about the awesome script abilities :)
19:58 arubi hopefully you'll defect. bitcoin is so much more fun than op_return ;)
20:25 gevs arubi, next time, dont ask for me to *defect* pff
20:25 gevs ill *detect* the right keys, yes :P and come back haha
20:25 arubi good luck :)
20:34 gevs one little thing though- related to OP_RETURN
20:34 gevs i am able to read integers out of it - but im guessing there must be some kind of charset to be encoded to too no?
20:35 eck there's no specified encoding, it's binary
20:35 gevs reading only integers, the only thing i could "read" out of it is that big endian signed + unsigned and little signed are all the same
20:36 gevs but little endian unsigned is different from those 3
20:36 arubi op_return is just an op code in a script. generally, your script would be "op_return 0x<data size push> 0x<data>"
20:36 gevs ah ok
20:36 gevs yeah omni defines different parts of that <data size to push>
20:37 arubi yea, I guess it's further encoded in the data part
20:37 arubi not sure what tether does, I'm guessing it's a bunch of varints :)
20:37 gevs 2 hexits Version, 2 hexits Type, 4 hexits property, 8 hexits value
20:37 arubi ah, so nothing like that
20:37 arubi seems simple enough to handle?
20:38 gevs the value is *kind of* 8 000 000 00 which makes it possible to be the real value - but there its not exactly 800000000 so i dont think that integer converted value is exactly the transaction value.
20:38 arubi what are you looking at?
20:38 gevs yes - it got me to understand the principle - and understand that this payload does not store anything about destination xD
20:40 arubi hexits, is that like "a 0-f character" ?
20:40 arubi if it's just a bunch of data being pushed and it's small enough, then there are special push operations
20:41 arubi [1,16] are [51,60]
20:41 arubi it's weird, maybe they're not using standard script stuff at all. I don't have an example to look at
20:41 arubi anyway, dinner time :)
20:45 gevs enjoy :)
21:16 laudiacay hiya! I'm trying to do analysis on the entire blockchain and spending patterns. I've got my bitcoin-cli installed, what would be the fastest way to get all the addresses that have ever participated in any transaction?
21:16 laudiacay i think just throwing them in a text file would probably be the best idea for now
21:17 eck you would call getblock for each block hash, it will return you the next block hash as well as the raw transactions
21:17 eck and then call gettransaction for each txn in each block
21:17 laudiacay oh there's like an API?
21:17 laudiacay where's the documentation?
21:17 eck so you can use bitcoin-cli itself, or there is a json rpc apik
21:18 eck which might be easier for what you're doing
21:18 eck bitcoin-cli help lists (most) of the api methods
21:18 laudiacay augh json makes me sad
21:18 laudiacay so it looks like i gotta connect first brb documentation binge
21:18 eck you could probably whip something up with bash and jq for what you're doing, but it might be easier to use a real library like https://github.com/jgarzik/python-bitcoinrpc
21:19 eck https://en.bitcoin.it/wiki/Original_Bitcoin_client/API_calls_list
21:20 laudiacay w o a h this is good
21:20 achow101 eck: don't use that list, it's outdated. just use the help command
21:21 laudiacay oh jeez this is so nice wow
21:56 gevs arubi - *after dinner* - anything i should look out for if i want to find a timelock function in the copay code? copay uses the bitcore js implementation ; i should be able to find some things in there for my multisig key problem
21:56 gevs tried a few key combinations and derivation paths but cant seems to find the 666 hash produced yet.
21:56 arubi there are no time locks. you're just missing the correct path
21:56 arubi gevs, the preimage to that hash is exactly 522102109c754588a5e6de512a68b0e82fa8ed6520f5fd57f6b41315e3d3c2b8f92ac2210318e0167d697e9366f17c2e83ac21c7e66ecc9427884d083887c8d3b6aa3857d8210350bc7cb1f4632c42ad092b1b825beea81ad4a663fa1c365c95d09ff44a11f30353ae
21:56 gevs ah ok cool lol
21:57 arubi try it out:
21:57 arubi `echo -n 522102109c754588a5e6de512a68b0e82fa8ed6520f5fd57f6b41315e3d3c2b8f92ac2210318e0167d697e9366f17c2e83ac21c7e66ecc9427884d083887c8d3b6aa3857d8210350bc7cb1f4632c42ad092b1b825beea81ad4a663fa1c365c95d09ff44a11f30353ae | xxd -r -p | sha256sum -b | xxd -r -p | openssl rmd160 -r`
21:58 gevs neat
22:14 jb55 btcs now has string and data literals, automatically get compiled to minimal pushdata https://jb55.com/s/9a68a136ac0710fe.txt
22:14 jb55 :]
22:28 gevs ha someone answered on BTT too ! with more insights about omni too :)
22:28 gevs fixes my understanding of how receiver and sender are determined!