Transcript for #bitcoin-dev 2014/02/10

00:33 copumpkin hmm, my computer froze and when I restarted it, bitcoin-qt complained that it couldn't read the block index and had to reindex the whole thing. I thought there was some reasonable database that didn't suffer from this backing the program?
00:34 andytoshi reasonable databases aren't fast
00:36 copumpkin I know we moved away from bdb, but can't remember what replaced it
00:36 emowataji leveldb
00:38 copumpkin hmm, and I guess that doesn't try to provide magic durability guarantees?
00:41 BB-Martino thx again for the help back there
00:53 sipa copumpkin: in theory, it does
00:53 kostagr33k can someone tell me why i get this after using bitcoind SENDMANY function: Warning! this transaction is a double spend of 112348699. You should be extremely careful when trusting any transactions to/from this sender.
00:53 kostagr33k i had about 72 outputs from 1 single input
00:54 sipa i'm sure it's not bitcoind telling you this
00:54 kostagr33k blockchain.info is
00:54 Luke-Jr ignore that website
00:54 Luke-Jr it is full of useless misinformation
00:54 kostagr33k that bad?
00:54 copumpkin sipa: :(
00:54 sipa copumpkin: which version?
00:55 copumpkin 0.8.5-beta
00:55 sipa git head / 0.9.0rc1 may improve things
00:55 Luke-Jr copumpkin: 0.8.6 fixed stuff like that :P
00:55 copumpkin gah
00:55 sipa on osx, 0.8.6 fixes some issues too
00:55 copumpkin why doesn't this thing have an update notification mechanism built into it yet? :P
00:56 sipa why do you think?
00:56 copumpkin cause someone hasn't implemented it yet
00:56 copumpkin ACTION touches his nose
00:56 sipa no, because it'd be one hell of a responsibility for the one that controls the update notification mechanism
00:56 Luke-Jr copumpkin: too much power
00:57 Luke-Jr Joe Update Deployment essentially owns the network then
00:57 copumpkin who runs the bitcoin-qt site or bitcoin.org?
00:57 Luke-Jr copumpkin: random people
00:57 sipa copumpkin: even if someone hacks the website, it doesn't cause an instant update of every node
00:57 Luke-Jr bitcoin-qt doesn't *really* have a site..
00:57 copumpkin sure
00:57 sipa but yes, the problem exists already
00:58 sipa there have been proposals to have auto-update, controlled by several developer's keys
00:58 Luke-Jr copumpkin: the long-term plan is to *offer* an update when N of M developers sign a release binary
00:58 copumpkin ah
00:58 Luke-Jr copumpkin: but that's new tech that isn't really done
00:58 sipa eh s/auto-update/update notification/
00:58 sipa i don't think i want auto-update at all
00:58 kostagr33k luke-jr: how long do I wait before i should be worried? any other site I can verify with?
00:59 Luke-Jr kostagr33k: worried about what? if you sent it, nothing to worry about..
00:59 kostagr33k luke-jr: i sure hope so
00:59 Luke-Jr other side can verify when it's 6 or 7 blocks deep in the blockchain
00:59 kostagr33k i just want 1 confirmation to pop in so i can feel at ease. i guess its smoke time
00:59 Luke-Jr kostagr33k: note that your txid *might* change in the process of this. txid is not guarateed to be the same
01:00 kostagr33k really? i thought the output from send many is the txid?
01:00 Luke-Jr it is
01:00 Luke-Jr but it could change
01:00 sipa kostagr33k: there's an attack that modifies transactions in-flight, which does barely more than annoy people and confuse some software
01:00 sipa kostagr33k: it doesn't actually steal or invalidate or double spend
01:00 copumpkin are people actively doing that?
01:00 sipa we've seen several reports the past few days
01:00 Luke-Jr copumpkin: ever since a certain well-known exchange was found to be vulnerable..
01:00 copumpkin lol
01:01 kostagr33k will bitcoin-qt show the new txid in the Transactions tab?
01:01 Luke-Jr kostagr33k: when it gets in a block, I think it will
01:01 sipa yes, it will show both
01:01 kostagr33k ok thanks luke-jr and sipa
01:02 kostagr33k 14 minutes and counting
01:02 sipa wait 2 hours
01:02 Luke-Jr I suspect the "original" txid/transaction will be a zombie unconfirmed forever
01:02 sipa yup
01:02 sipa or confirm :)
01:02 Luke-Jr sipa: I wonder if we should hold 0.9 off for a fix to that, since it seems someone is automatically doing it all the time now..
01:03 Luke-Jr well, if the original confirms, the mutation will never show
01:03 copumpkin anyone know more about the O((n^2)!) algorithm?
01:03 jaakkos do the openssl devs care?
01:03 kostagr33k so i guess i guess sit and wait? i was going to e-mail the end users but not if the TXID could be FUD
01:03 copumpkin jaakkos?
01:03 jaakkos about the malleability
01:04 copumpkin I thought it was just a bitcoin issue, not including txid in signature
01:04 copumpkin or other metadata, I guess
01:04 gmaxwell copumpkin: I was talking about the recursive ismine() check for confirmed inputs on unconfirmed transactions, I think it's factorial complexity, the n^2 is a bit of an an exageration.
01:05 jcorgan sipa: can i get your help with something?
01:05 copumpkin gmaxwell: interesting! is there a writeup of that anywhere or do I just need to dig into the code?
01:05 kostagr33k luke-jr: ok i checked one of the addresses and I see the payment .. ill sit and wait. Thanks for the help as well sipa
01:05 jaakkos what is the proposed fix to the malleability issues?
01:05 gmaxwell copumpkin: phantom wrote about it.
01:06 gmaxwell jaakkos: it won't be fixed for a long time, services need to be robust against it.
01:07 jcorgan sipa: i have your 'addrindex' branch *almost* compatible with current master
01:08 jcorgan it's current up to Dec. 20, but there is a merge conflict with current master since then that should be straightforward, but I can't quite resolve it
01:08 jaakkos yeah, i see it can't be changed...
01:16 sipa jcorgan: ok?
01:17 jcorgan so i was hoping you could do a trial merge of current master into my branch, and either resolve or help me to resolve the conflict in main.cpp and txdb.cpp
01:17 sipa not now
01:18 sipa i expect the largest problem is the changed leveldb wrapper interface
01:18 jcorgan that's all taken care of
01:18 sipa ok, so what's the problem?
01:18 jcorgan i've been incrementally merging master since you branched off, and fixing up the merge conflicts commit by commit
01:19 jcorgan the conflict is actually quite small now
01:19 sipa sounds painful
01:19 jcorgan it has taken hours
01:19 jcorgan but i've got a addrindex branch on my github, and right now there's a node reindexing its database with it for testing, but the last master merge is from dec. 20
01:21 jcorgan i think it is just one problematic commit away from being a ready for testing
01:21 jcorgan i can pastebin it if you want
01:25 jcorgan http://pastebin.com/SuYr8eQu
01:25 jcorgan this particular conflict was introduced in 96e5f61d
01:26 jcorgan anyway, i understand if you don't have time right now
01:27 sipa yeah, that's the encapsulated leveldb iterator
01:27 sipa i might PR that independently from addrindex at some point
01:29 jcorgan well, so far the reworked addrindex branch i have is working against master @ 086d7ec2, i just need to test the RPC interface
01:29 jcorgan (that's Dec. 24)
01:38 aynstein WilsonNL: what are u compiling?
01:40 zapsoda Whats the proper way to convert BTC to satoshi (because a function needs intergers) and back again using python? Is it just divide and multiply by 100,000,000? Should I store the BTC values as floats?
01:41 jcorgan zapsoda: no
01:41 jcorgan store them as integer satoshi's and convert for display
01:44 zapsoda Ok, thanks, And to get them to satoshi do I multiply by 100 million?
01:44 zapsoda satoshi's(
01:44 zapsoda And for send functions I need to convert them back to *floats*?
01:44 zapsoda or decimals?
01:44 zapsoda sorry, noob here :p
01:49 emowataji pity the send and display functions in the rpc interface don't use integer satoshis
01:50 sipa yes, very much so
01:50 Luke-Jr zapsoda: round(btc * 1e8)
01:50 zapsoda Thanks Luke-Jr
01:50 Luke-Jr sending, prob best to use Decimal type and /1e8
01:52 zapsoda Luke-Jr, I'm getting TypeError: unsupported operand type(s) for *: 'Decimal' and 'float'
01:53 zapsoda Oh, Does your second reply have something to do with that :p
02:00 zapsoda Oh I see, You were saying for sending it would be best to use decimal type and divide by 1e8
02:00 zapsoda And idea why I'm getting unsupported operand type(s)/
02:10 Cryo ^
02:20 phantomcircuit sipa, the last i heard you were writing a patch to remove vtxPrev
02:20 phantomcircuit is that still true?
02:20 sipa phantomcircuit: lost track of that
02:21 sipa feel free to bring yours up to date
02:21 sipa i think i've made comments there
02:30 gmaxwell https://bitcointalk.org/index.php?topic=457546.msg5034918 weee1
02:30 sipa that's the 3rd one today here
02:30 sipa someone is running a malleability bot, i think
02:31 andytoshi sipa: thx for the malleability writeup
02:32 sipa yw
02:34 fanquake ACTION wonder why people don't upgrade their clients
02:35 petertodd sipa: keep in mind that there may be some very deep malleability issues with ECC sigs themselves
02:36 petertodd sipa: I think we need to separate malleability that is annoying because of poorly coded wallet software and malleability that fundementally makes multi-party protocols dangerous - the latter can IMO be better fixed with a new CHECKSIG type
02:36 gmaxwell petertodd: well we've enumerated the only one we know of, and be darned we certantly tried to find another algebraic manipulation.
02:37 petertodd gmaxwell: last I heard from adam he still wasn't sure; has that changed?
02:37 petertodd gmaxwell: I'd just hate to see people rely on "fix malleability via whack-a-mole" and have that fail
02:38 sipa apparently people have relied on non-malleability for a long time already
02:38 gmaxwell he's not sure, I'm not sure. To be sure we'd need to find a proof that rerandomizing a DSA signature lets you solve discrete logs.
02:38 petertodd gmaxwell: sounds like a lot of not sure :)
02:38 andytoshi ACTION is also not sure ;)
02:38 petertodd sipa: well, people rely on zeroconf being safe so...
02:39 gmaxwell on the other hand, we spent a couple hours futzing with algebra and didn't find a way to do it.
02:39 petertodd gmaxwell: yeah, i still haven't broken the block cipher I made, must be good stuff :P
02:40 phantomcircuit gmaxwell, bc.i doesn't show any difference between them
02:40 phantomcircuit amusing
02:40 sipa phantomcircuit: one of the pushes is changed to a OP_PUSHDATA4
02:40 phantomcircuit ah that explains why they appear identical
02:56 zapsoda Luke-Jr, when I convert them to decimals and divide by 1e8 I get numbers with alot of decimal places, Would I have to round down to the nearest 8th decimal place? Or is there a better way to do this?
02:58 Luke-Jr zapsoda: convert to Decimal before you divide
02:59 maaku zapsoda: did you divide by 1e8 or Decimal(100000000)?
03:01 jaakkos imo the whole mtgox mess just shows how damn difficult it is to come up with bitcoind-compliant implementation
03:01 jaakkos would it make sense to separate the core functionality into a library and encourage everyone to use it in their wallet implementations?
03:02 jaakkos anyone implementing their own wallet from scratch seems to be screaming for trouble
03:02 jaakkos no matter how good you are
03:02 andytoshi they weren't good
03:02 jaakkos :D
03:03 maaku jaakkos: doesn't mtgox use (ancient, archaic, heavilly modded) bitcoind?
03:03 jaakkos i don't know anything else but just keep hearing it's "custom"
03:04 zapsoda Luke-Jr, I was converting the divided value to decimal but when I convert the int to decimal first then divide by 1e8 I get
03:04 zapsoda TypeError: unsupported operand type(s) for /: 'Decimal' and 'float'
03:07 sipa jaakkos: not sure what you understand under 'core' functionality, but in my view, it certainly excludes the wallet
03:08 jaakkos perhaps i wanted to say node
03:11 jaakkos oh well, not sure how to go about it. i'm just trying to think what could be done to avoid such issues in the future.
03:12 sipa i don't think it's really an implementation issue
03:12 sipa the problem is infrastructure depending on an incorrect assumption
03:14 jaakkos the assumption being that coins are not spent if the gox-generated tx isn't in ledger?
03:14 jaakkos gotta stop thinking about it and wait for the announcement :)
03:14 sipa announcement of what?
03:14 jaakkos the one gox is supposed to release today
03:14 phantomcircuit sipa, they put in place a self imposed timeline of an annoucement on monday
03:15 sipa yeah sure
03:15 sipa but i'm not expecting anything interesting there
03:15 phantomcircuit jaakkos, at the time mtgox implemented their own client they didn't have a lot of great options for handling the volume of payments they were supporting
03:15 sipa either it will be "still too much problems" or "withdrawals possible again"
03:15 sipa i assume
03:16 jaakkos i don't even have funds at gox but i'm just very much interested in hearing what the problems are
03:16 phantomcircuit jaakkos, giant accounting mess would be my guess
03:16 sipa jaakkos: http://www.reddit.com/r/Bitcoin/comments/1x93tf/some_irc_chatter_about_what_is_going_on_at_mtgox/...
03:17 phantomcircuit fixing the underlying problem should be relatively simple, but the giant tangle of thousands of cross referenced transactions isn't
03:17 jaakkos sipa: yeah, this i saw, i actually asked:
03:17 jaakkos > i wonder if someone can show a set of 3 txs such that: 1) tx that attempts to double spend output X, 2) tx that spent X but signature was malformed and 3) modified tx 2 that was accepted in ledger that really ended up spending X
03:18 jaakkos i had the feeling that would give ground to gmaxwell's theory?
03:29 zapsoda Finally got the values to round down using:
03:29 zapsoda But now I'm getting "TypeError: Decimal('0.00300986') is not JSON serializable" any ideas?
03:29 zapsoda Decimal(amounts[i] / 1e8).quantize(Decimal(10) ** -8, rounding=ROUND_DOWN)
03:34 phantomcircuit zapsoda, the decimal object isn't serializable
03:35 phantomcircuit you need to convert to a float and/or write an encoding routine for decimal.Decimal objects
03:35 zapsoda float seems alot easier but wont I lose to much precision?
03:36 zapsoda If you need to encode decimals to use them in RPC there must be a already made function for bitcoin right?
03:36 phantomcircuit zapsoda, with standard floating point you will lose no precision
03:36 phantomcircuit zapsoda, check jgarzik's github
03:36 phantomcircuit python-bitcoinrpc
03:36 phantomcircuit or maybe it's in python-bitcoinlib now
03:36 phantomcircuit i cant remember
04:56 fanquake Finally found a decent mirror to download qt 5.2.1
04:56 fanquake Half of the available mirrors seem to be down, or want me to wait a good 10 hours..
05:00 benkay if anyone in here has some java experience I'd appreciate a bit of help reading something that's totally opaque to me: https://code.google.com/p/bitcoinj/source/browse/core/src/main/java/com/google/bitcoin/core/Utils... i get that we're stepping through a 4-byte-wide byte array and setting each byte individually but this hairy (0xFF & (val >> 24)) stuff is breaking my poor brain
09:22 abrkn what's error code -4 when submitting a tx using json api?
09:27 wumpus huh, -4 is RPC_WALLET_ERROR an 'unspecified wallet error', it shouldn't happen while submitting a tx
09:28 wumpus or at least, what command are you using?
10:03 gmaxwell ::sigh:: https://www.mtgox.com/press_release_20140210.html
10:12 wumpus indeed, sigh
10:14 airbreather trying to understand... can't MtGox just scan each block for transactions that spend the same inputs as the transactions they created, rather than one that matches by hash?
10:14 sipa they should do that
10:14 Apocalyptic <SarahCoinBit> ibtc The issue is much bigger than Mt.Gox, it affects all bitcoins
10:14 Apocalyptic such fud...
10:14 wumpus airbreather: of course they can
10:14 sipa but the problem is apparently services where people use the txid to prove a transaction from them didn't confirm
10:14 sipa which is broken
10:14 airbreather ahh
10:15 wumpus but they like to call it a protocol bug because that allows them to pretent they're not responsible
10:15 sipa admittedly, no wallet client deals well with this situation afaik
10:15 Apocalyptic indeed sipa
10:15 Blitzboom gmaxwell: do they speak the truth?
10:15 Blitzboom is it on the devs to change this?
10:15 wumpus and yes maybe it is a bug that transaction malleability is allowed, but they were generating transactions that were not accepted by the network that's their problem
10:16 Apocalyptic Blitzboom, are you serious ?
10:16 wumpus Blitzboom: reducing the maleability was already under way a long time
10:16 sipa Blitzboom: he has talked to us in private, and we suggested some things, but i certainly wasn't aware of them relying on us to get the whole bitcoin ecosystem on this
10:16 Apocalyptic why should the dev change the way mtgox check if the tx did or did not go through ?
10:16 gmaxwell Blitzboom: It is exceptionally unlikely that malleability will be fixed in the year. We've been slowly moving towards improving it, v0.8 closed of many forms of it.
10:17 Blitzboom wow
10:17 Blitzboom mtgoxBTC are fucked?
10:17 sipa no
10:17 gmaxwell Blitzboom: no.
10:17 wumpus no, they're not, they just need to change *their* software
10:17 gmaxwell They just need to change how they identify transactions.
10:17 Blitzboom doesn't sound like they intend to do that
10:17 Blitzboom but thanks
10:17 gmaxwell They're also talking about having another kind of txid so customers could look up normalized transactions.
10:18 torokun gmaxwell: why do they have to wait for a standard to implement their own unique txn hashing?
10:18 torokun couldn't they just do it their own way?
10:18 sipa anyway, if they want an approved and implemented standard: i suggezted this: compute the sighash of all of a tx's inputs (while setting the prevout script to []), and hash all these sighashes together
10:18 wumpus Blitzboom: I'm sure they're working on that very hard right now
10:18 gmaxwell I'm a little disinclined to spend a lot of resources on that, because it doesn't solve a bunch of the other problems malleability presents, like griefers killing chains of unconfirmed txn.
10:18 torokun it's just a matter of accounting...
10:19 gmaxwell sipa: otoh, it's not something thats free to support as a first class ui feature, e.g. lookup by that ID would require another large index.
10:21 sipa what about some python tool checked into the repository that can calculate the normalized txid for a raw transaction?
10:21 gmaxwell doesn't really address the "where is my transaction? this txid you gave me matches nothing!" case though
10:22 sipa yeah, but what does adding it to the bitcoin core wallet help? i doubt few involved services use tbat wallet
10:22 sipa it'd be easy to add though
10:26 orweinberger gmaxwell Doesn't mtgox's announcement contradicts your technical explanation?
10:26 airbreather I dunno... it seems to me that this is a social problem rather than a technical one. Worst-case, MtGox support gets flooded with people who try to scam them this way, and then after a trivial amount of scanning, they can verify that this is going on and then ban those customers.
10:27 xeroc basically: mtgox accounting software is broken .. other exchanges not necessarily broken, too?!? is that right?
10:28 dandroper seems unclear atm
10:28 gmaxwell orweinberger: not really. It makes some implications which are just flat out untrue.
10:29 gmaxwell E.g. that this is something new.
10:29 orweinberger So it's possible they are lying to buy some more time..
10:29 gmaxwell Rather than something we've known long enough about that there are posts in 2011 about it and a page on bitcoin.it wiki.
10:29 gmaxwell I don't think they're lying.
10:29 gmaxwell but who knows.
10:29 wumpus indeed, that's what they skip over: why does mtgox suffer from this so badly, while other exchanges continue functioning as normal
10:30 comboy but it would be nice to have a reliable handle to transaction in general, I mean when nobody reuses addresses it is solved I guess, but until then..
10:30 wumpus blame it on the protocol, why not
10:30 dkog "We have discussed this solution with the Bitcoin core developers and will allow Bitcoin withdrawals again once it has been approved and standardized." ... WHAT ?!? I wish I was in a smaller room so I could bang my head against more walls at once...
10:30 orweinberger They're saying this is a 'weakness' in the bitcoin protocol and not specifically related to MtGox, if that's true why aren't we seeing these issues on Bitstamp or any other exchange
10:30 gmaxwell I was surprised by the tone and approach of their announcement. There are support related issues why they'd want something to be externally adopted but nothing I know or in the press releases suggests a reason that their withdraws need to be held until that happens.
10:30 Apocalyptic orweinberger, it's not true... ffs
10:30 orweinberger Apocalyptic, I agree
10:31 sipa the only problem is relying on txids to remain unique
10:31 Apocalyptic ^
10:31 gmaxwell orweinberger: my explaination was 1/4 about that, it's the stuff at https://en.bitcoin.it/wiki/Transaction_Malleability
10:31 comboy their dummy support people are not able to verify transaction in any meaningful way probably other than copy pasting txid in their internal checker, so they probably need to fix that
10:31 gmaxwell comboy: sure but is that a reason to hold all withdraws entirely?
10:32 gmaxwell and thats not up to any bitcoin core developers, for dummy compat they need bc.i buy in.
10:32 comboy gmaxwell: not for me, I guess they have been scammed and need to clean up before proceeding
10:32 dkog I don't understand why they can't just hash transactions however they want.
10:33 gmaxwell dkog: they can.
10:35 dkog So... why couldn't they just hash on txin, or the signatures themselves, .. I don't get why they make it out like it's impossible
10:36 gmaxwell dkog: they want some standard string fools can look up on bc.i, I still can't understand why its gating.
10:36 airbreather other than the output address, of course
10:36 airbreather ...
10:36 orweinberger gmaxwell, do you think this entire issue can be solved solely by mtgox by implementing their withdrawal process in a different way and without having to change anything in the bitcoin protocol? (I'm guessing the answer is yes, since other exchanges are able to do so)
10:37 sipa this has nothing to do with the protocol
10:37 airbreather they can just implement their own nonstandard hashing scheme and expose it in a web service
10:37 dkog gmaxwell: are you reading between the lines or have other info? I don't see them say outright that's what they want
10:37 dkog orweinberger: yes!
10:38 gmaxwell dkog: somewhat both. I know they want that, I don't know if thats what they're reffering to as gating. If so I can't figure out why.
10:38 maaku orweinberger: yes
10:38 airbreather "here's some numbers, go to http://mtgox.com/wheres-my-money and plug them into that box to track your withdrawal"
10:38 Plarkplark Mtgox statement: Can the bug block a transaction forever? Or just a while to fool the sending party? What is the time and computing power required to block longer then 10 minutes?
10:39 maaku orweinberger: instead of reporting only the hash to the user, they should record, report, and track the transaction itself
10:39 airbreather hell, they could even make "some numbers" be the original txid, and implement a translation scheme that's resistant to this behind the scenes if they want
10:40 dkog IN related news, Amazon refuses to ship any packages to Florida until the state builds a dome over it to prevent rain from damaging the cardboard boxes that are left uncovered in front of people's houses...
10:44 gmaxwell dkog: I'm at least glad to see your comments. :)
10:44 dkog it was a real conversation killer :)
10:45 coffeecoco dont be shy due to me
10:45 coffeecoco do tell!
10:47 maaku gmaxwell: nice writeup on reddit, thank you
10:48 Plarkplark gmaxwell: +1
10:50 nkuttler link?
10:51 midnightmagic nkuttler: http://www.reddit.com/user/nullc and check comments, backwards.
10:51 nkuttler midnightmagic: oh, that one. thanks :) was confused because of the nick
10:51 midnightmagic yah
10:54 maaku for others who might be lazy: http://www.reddit.com/r/Bitcoin/comments/1x93tf/some_irc_chatter_about_what_is_going_on_at_mtgox/...
10:58 thermoman is there a problem with BTC transactions?
10:58 thermoman because we had a problem some hours ago
10:58 thermoman and now mt gox says it has problems with transactions
10:59 cazalla are the mtgox claims true?
11:00 sipa depends on how you interpret them
11:00 thermoman is there something wrong with the official bitcoin client?
11:00 thermoman 0.8.6
11:00 thermoman or the protocol?
11:00 thermoman can i resume operation or should i suspend?
11:01 sipa neither has any problem, and neither will be changed
11:02 sipa the problem is in infrastructure apparently making the assumption that transaction ids cannot change - something which has been known for years to be not true
11:02 sipa mtgox refers to a suggested standard for dealing with this
11:03 maaku thermoman: there is nothing wrong if you are following standard accepted practices
11:03 jddebug Still keeping the same sha256 or would mining change?
11:04 maaku jddebug: ?
11:04 sipa jddebug: NOTHING WILL CHANGE
11:04 airbreather (does this happen every time someone spreads FUD?)
11:04 sipa there is nothing wrong with the protocol
11:05 wumpus no, mining has nothing to do with this at all
11:05 wumpus where do people get those idiot ideas?
11:05 coffeecoco fear^
11:06 jddebug Hey. Was asking if thats what was meant by their rederal to change in protocol. Geesh
11:07 jddebug *referal
11:07 sipa jddebug: sorry, we're kinda overwhelmed by that statement
11:07 jddebug Kk
11:07 wumpus before you ask question about specific details that make no sense, I suggest you try to understand the big picture
11:10 midnightmagic wumpus: Knowledge hierarchies which go through minds that don't understand bitcoin, but answer questions for scared people. It's normal, it's going to continue happening too.
11:10 michagogo cloud|wumpus: Why do I keep getting emails from pulltester?
11:12 Wild0wnes whoever killed the gmaxwell dialog in here i hate you
11:12 wumpus michagogo|cloud: probably because you are subscribed to some pull request
11:12 michagogo cloud|wumpus: Yes, your two gitian PRs
11:12 wumpus michagogo|cloud: there's an unsubscribe button
11:13 michagogo cloud|Are you updating them or something?
11:13 wumpus yes
11:13 michagogo cloud|Ah
11:13 michagogo cloud|(with what?)
11:13 wumpus still trying to get things deterministic
11:13 michagogo cloud|I don't mind, I was just wondering what was changing, since there didn't appear to be a change on the page
11:14 wumpus I promise I'll give up today if it's still not going to work
11:14 michagogo cloud|Who are you promising that to? :P
11:15 wumpus I think even the .tar.gz is deterministic now, though
11:16 gmaxwell I'm hearing from a lot of people who are asking things like if it's safe for them to send bitcoins now.
11:16 midnightmagic whoah how did you do that? I couldn't get deterministic tar even by manipulating datestamps in the origin files.
11:16 wumpus gzip even includes a timestamp field in its header, isn't that wonderful
11:17 wumpus midnightmagic: file names through a sort, normalize all permission/uid/gid/timestamp information, and tell gzip to not add a timestamp
11:17 UukGoblin maaku, what writeup on reddit?
11:17 warren wumpus: oooooh
11:17 maaku UukGoblin: http://www.reddit.com/r/Bitcoin/comments/1x93tf/some_irc_chatter_about_what_is_going_on_at_mtgox/...
11:17 wumpus midnightmagic: tar -xvf $HOME/build/bitcoin/$DISTNAME | sort | tar --no-recursion -cT /dev/stdin --mode='u+rw,go+r-w,a+X' --owner=0 --group=0 --mtime="$REFERENCE_DATETIME" | gzip -n > $OUTDIR/src/$DISTNAME
11:18 UukGoblin maaku, thanks
11:19 midnightmagic wumpus: Awesome, thanks.
11:22 davout fun fact: i just deposited to my bitcoin-central account and it appeared twice under two different TXIDs, so apparently someone is out there, randomonly mutating TXIDs
11:22 CrackRockerson ghosts
11:23 davout dem leprechauns
11:23 randy-waterhouse davout: fun fact : Instawallet funds disappeared without a trace
11:23 randy-waterhouse leprechauns?
11:24 randy-waterhouse maybe they appeared in your bitcoin-central account twice already?
11:25 davout randy-waterhouse: this is -dev not -kindergarten
11:25 randy-waterhouse right you better leave
11:26 airbreather davout: https://bitcointalk.org/index.php?topic=8392.msg122410#msg122410 it's Enochian's echo bot!
11:26 gmaxwell sipa http://www.nu.nl/internet/3697861/bitcoin-weer-onderuit-lek-in-software.html
11:26 sipa gmaxwell: see my mail
11:27 wumpus goddamnit
11:27 davout airbreather: did the guy actually make a bot??
11:27 davout first time i witness this
11:27 airbreather no, probably not, just a tongue-in-cheek comment
11:28 sipa it would not surprise me that some bot doing this is actually out there
11:28 davout hehe, i suppose some people would find an interest in doing just that
11:28 sipa there have been many random cases
11:28 davout sipa: yeah, the profit opportunity is pretty clear
11:28 airbreather some people just want to see the world burn. or at least, scramble around a bit confused, asking if transactions still work
11:29 Aahzmundus Stupid since news like this sets back mass adoption a good few months
11:29 davout airbreather: or crash&buy
11:29 maaku davout: more charitably, it seems that mtgox was making transactions with incorrect encodings, and it may be that someone was just 'helpfully' fixing these after they became non-standard in 0.8
11:29 maaku but i don't think we know yet
11:31 davout maaku: well, as long as they were going through they were valid, remember that the "standard" thing is not in the protocol per se
11:31 jouke gmaxwell: misinformed journalist, we'll contact them right away
11:31 Plarkplark jouke: dutch?
11:31 jouke yes
11:31 Ademan Trying to get out of the storm... so... regarding scriptSig maleability, are there any other implications for that, or just, again, invalidating the transaction hash?
11:31 thermoman "As a result the Mtgox wallet believed some coins were available for spending which really had already been spent and it began double spending those inputs"
11:31 Plarkplark Because the bigger news sites will pick this up asap in NL
11:32 sipa Ademan: s/invalidating/changing/
11:32 Ademan sipa: sorry, right
11:33 airbreather Ademan: any transaction that spends the outputs from the losing txid also becomes invalid once the winner gets a few confirms, but that's as far as I think it goes...
11:33 Ademan I guess I said invalidating since other things depend on the hash, but really it's just changing
11:33 sipa Ademan: the problem is the definition of validity here... if you have a double spend or other forms of conflicting transactions... both are valid
11:34 sipa Ademan: it's just that only one will be accepted
11:35 wallet42 how exaclty can i change the hash of a signed tx?
11:35 gwb3 sipa and gmaxwell - if there is anything we can all do to help support your efforts now, more than ever, please do let us know
11:35 UukGoblin so having read all this, I've come to the conclusion that there's nothing wrong with bitcoin itself, and that you shouldn't trust unconfirmed transactions, and also that you shouldn't rely on the txid not changing
11:35 airbreather +1 UukGoblin
11:35 Ademan wallet42: https://en.bitcoin.it/wiki/Transaction_Malleability
11:36 sipa UukGoblin: bingo
11:36 wallet42 ademan:thx
11:36 Ademan wallet42: np, please don't try this at home ;-)
11:37 gwb3 there are two issues now, 1. a technical issue at mt.gox 2. a pr problem for bitcoin
11:37 sipa gwb3: thanks, but i don't think it's up to us to fix things
11:37 gwb3 sipa: np ;)
11:38 Ademan gwb3: maybe they were hoping after this nobody would want their bitcoins any more and they would be off the hook for their insolvency ;-)
11:38 UukGoblin gwb3, well, also probably 3. another technical issue at mtgox with cleaning all the mess up
11:39 gwb3 i will continue to spread positive news items on social media to fight the good fight
11:39 UukGoblin I don't expect it to be fixed any time sooner than their bank withdrawal problems ;-)
11:40 Plarkplark Why does he say this
11:40 Plarkplark The problem we have identified is not limited to MtGox, and affects all transactions where Bitcoins are being sent to a third party. We believe that the changes required for addressing this issue will be positive over the long term for the whole community.
11:40 money https://www.mtgox.com/press_release_20140210.html The problem we have identified is not limited to MtGox, and affects all transactions where Bitcoins are being sent to a third party. We believe that the changes required for addressing this issue will be positive over the long term for the whole community.
11:41 money whats that
11:41 money he sais the whole bitcoin network has a bug
11:41 UukGoblin no
11:41 sipa it does not
11:41 money it's just not onl y him
11:41 UukGoblin he says that other exchanges might have similar bugs as mtgox
11:41 Ademan Plarkplark: It's not clear to me why this doesn't affect (unconfirmed transactions) in regular wallets
11:41 money it does yes read, in black andwhite
11:42 money don't sweetned this, is it a bitcoin problem or not?
11:42 UukGoblin money, it is not
11:42 Ademan ACTION needs to finish reading the source code so he knows how transactions are indexed
11:42 Plarkplark bitcoind seems uneffected, according to gmaxwell
11:42 money so why did he write that nonsense if it's nonsense
11:42 Plarkplark So is this like a "wear your seatbelt" kinda thing. and gox did not listen?
11:43 randy-waterhouse money: it's an edge case if you are a large target like an exchange doing lots of transactions without confirmations
11:43 Plarkplark and others _might_ als not be listening to the devs?
11:43 money gmaxxell should openly disqualify his statements.
11:43 money but well, that means that devs have to work on scalebility right?
11:43 wallet42 what statement?
11:43 money so it does have limitations.
11:44 sipa money: it has nothing to do with scalability
11:44 randy-waterhouse money: gox-like enterprises have limitations, but we already knew that
11:44 money what is it then?
11:44 thermoman If an exchange uses the official bitcoind and accounts incoming BTC to user accounts only if at least 6 confirmations were seen on the network for this specific TXID - is this exchange safe or not?
11:45 sipa money: bitcoin infrastructure (not the protocol or wallets themselves) assuming that transaction id's are immutable
11:45 money well so what he says isn't making sense? bitcoind is alright? or do you have to expand it? may other exchanges get into the same problem? btchina, houbi, and other exchanges?
11:46 Plarkplark money: only if they dont handle tx properly
11:46 maaku money bitcoind is fine
11:46 money I see.
11:46 Ademan thermoman: I believe the issue actually lies on the outgoing side, but yes, I believe there is no problem with sufficiently confirmed transactions
11:46 Plarkplark maaku: so there was a bugfix years ago for this problem in the official client?
11:46 maaku Plarkplark: there isn't a problem in the reference client
11:47 wallet42 bitcoind is alright but it currently introduces a trapdor if you dont "get" the possible issues in your local database scheme
11:47 money well why is mtgox blaming it on the outher external circumstances.... I know mtgox is bad, but there could be a possibility. And doesn't bitcoin produce standard wallets that work perfectly well with the bitcoinds?
11:48 wallet42 well mtgox is right about that this "trapdrop" one may fall trough if you dont pay attention to some edge cases could be closed forever
11:50 money Bitcoin transactions are subject to a design issue that has been largely ignored, while known to at least a part of the Bitcoin core developers and mentioned on the BitcoinTalk forums. https://www.mtgox.com/press_release_20140210.html <--- he is attacking the system.
11:50 money "Largely Ignored"
11:50 money DA NERVE.
11:50 starsoccer So are the bitcoin devs activily looking for a solution/working on a solution or its not a huge rush
11:50 wallet42 it only affects haevy bitcoin transaction producers, that rely their check "this money was send?" on the incorrect assumptin that a txhash is not malleable
11:51 Plarkplark starsoccer: solution: use the reference (bitcoind) client. done.
11:51 Ademan maaku: why isn't this a problem for unconfirmed transactions for all clients?
11:51 starsoccer Plarkplark, yes i know that lol
11:51 randy-waterhouse who woulda thought ... the centralised users of decentralised system are having problems?
11:51 Apocalyptic starsoccer, it's not up to bitcoin devs to work out a sol for mtgox
11:51 starsoccer i just mean so the devs arent changing the whole protocol for mtgox
11:51 starsoccer oo i know
11:51 sipa the first e-mail on the bitcoin-dev mailing about malleability is from me, send on october 20th, 2012
11:51 Plarkplark If a car is broken, do you fix the road?
11:51 maaku starsoccer: there is no problem with the protocol or with bitcoind
11:51 starsoccer just mtgox has an interesting way of blaming others for its problems
11:52 maaku nothing is broken
11:52 maaku except mtgox's accounting
11:52 starsoccer lol
11:52 sipa maaku: apparently not only theirs
11:52 starsoccer yes just mtgox trying to push out some blame
11:52 maaku sipa: true
11:52 wallet42 sipa: what is your take, should the protocol be changed?
11:52 starsoccer thats what i figured
11:52 sipa wallet42: no
11:53 Ademan sipa: why not? Surely having reliable txids even for unconfirmed transactions is *desirable*
11:53 sipa wallet42: there are some changes that can be made to slowly root out the problems caused by malleability
11:53 wallet42 sipa: so better local implementation is the answer
11:53 maaku Ademan: any improperly coded wallet application could suffer this problem. i don't know what other clients are vulnerable
11:53 sipa wallet42: but that will take years, is not a guaranteed fix for every potential issue with it, and is not really the issue here
11:54 sipa Ademan: right, if we'd design bitcoin from scratch today, we'd make sure malleability was of no concern
11:54 sipa but there is a huge infrastructure that cannot just be changed
11:54 money oh
11:55 Ademan maaku: isn't this a problem for bitcoind too? Unconfirmed transactions' txids may not be what the originating client expects, how could that *not* be a problem?
11:55 starsoccer sipa so years, like gmaxwell said. thats what i ifigured
11:55 sipa Ademan: yes, bitcoind's wallet doesn't deal with it particularly well
11:55 maaku Ademan: because txids are meaningless
11:55 sipa Ademan: only when you accept unconfirmed transactions, though
11:55 davout maybe they should be re-labeled to something else than "IDentifier" then :-)
11:56 Plarkplark I hope the devs come up with a good, short post on what/how/when/who.
11:56 money so if I get this malleability concept correctly, some otherguy can use up coins, and actually doesn't need to own them?
11:56 maaku the reference wallet in bitcoind annoyingly doesn't handle mutant transactions very well
11:56 gmaxwell Your site says it payed a customer in txid X. They call bitching, and the tx is unconfirmed for N weeks. Your support staff looks and indeed txid X is unconfirmed for weeks. ... problem there is that the _only_ safe thing to do is to respend and keep the same inputs. Mallability is irrelevant to the safty in this case. So thats a bit interesting.
11:56 maaku but you don't lose money because of it
11:56 sipa maaku: NO
11:56 Plarkplark Get this finger pointing out of the way.
11:56 sipa eh
11:56 Ademan money: no
11:56 sipa money: NO
11:56 sipa money: all that can happen is that someone can modify a transaction's txid
11:56 sipa money: without changing the source or destination of the money
11:56 sipa money: and without invalidating it
11:56 wallet42 money: no you can cange txid
11:56 Plarkplark gmaxwell: can they delay the tx for weeks?
11:56 money ok
11:56 gmaxwell Plarkplark: no, why are you asking me this again?
11:56 Plarkplark sorry
11:57 maaku money: the transaction is *exactly* the same ... just with a different hex id
11:57 money i see
11:57 wallet42 but if you store in your DB: user_x got paid by TXID
11:57 wallet42 and now there is TXID' in the block
11:58 gmaxwell Plarkplark: mtgox's delayed transactions are because they were spending already spent coins, and before that because they were producing invalid der encodings, and before that because they were spending immature coins. The existance of mutation doesn't give anyone a way to delay payments that I'm aware of.
11:58 wallet42 the user got his funds
11:58 wallet42 but in your DB the TXID is marked as not included in block
11:58 gmaxwell maaku: say functionally the same instead of exactly. :P
11:58 maaku gmaxwell: yes, thank you
11:58 Hawkix Is there a way to verify ex-post that the immutable hash id were indeed misused by someone?
11:58 Plarkplark gmaxwell: Ok so the explenation in the press release does not really match what we saw in the blockchain?
11:59 gmaxwell Plarkplark: which part?
11:59 maaku Hawkix: yes
11:59 Plarkplark the "bug"
11:59 Ademan so is the way to "account" for this correctly to compute your own "id" based on unmaleable portions (a somehow normalized signature and scriptsig + inputs+outputs) ?
11:59 Hawkix maaku: by searching for additional "padding spam" in the mined transactions?
11:59 gmaxwell Ademan: sure, thats a way of accounting. Or you track spentness on the basis of which inputs were used.
12:00 maaku Ademan: that would be an ideal thing to do, yes, although theoretically we don't even know if that is possible
12:00 gmaxwell 03:56 < gmaxwell> Your site says it payed a customer in txid X. They call bitching, and the tx is unconfirmed for N weeks. Your support staff looks and indeed txid X is unconfirmed for weeks. ... problem there is that the _only_ safe thing to do is to respend and keep the same inputs. Mallability is irrelevant to the safty in this case. So thats a bit interesting.
12:00 gmaxwell IMO inputs are necessary anyways, see my above example:
12:00 wallet42 is the upcomming "relay double spent" transactions a possibly mitigation
12:00 starsoccer gmaxwell, so basically mtgox is just trying to push the blame onto you when its their custom software
12:01 wallet42 ?
12:01 gmaxwell E.g. imagine the tx was just delayed and never mutated. You reissue using different inputs? you're potentially screwed, since the original tx might eventually go through. To be safe you must reuse the same inputs, and then you don't care about mutation.
12:01 gmaxwell starsoccer: well the world is not as simple as A vs B. Bitcoin has some quirks which make correct usage harder than I wish it were.
12:01 Ademan makes sense
12:02 starsoccer gmaxwell, yes, for sure, but i just mean they seem to like to say the bitcoin devs need to fix it rather then we will just adjust our software
12:03 money We have discussed this solution with the Bitcoin core developers and will allow Bitcoin withdrawals again once it has been approved and standardized. In the meantime, exchanges and wallet services - and any service sending coins directly to third parties - should be extremely careful with anyone claiming their transaction did not go through.
12:03 starsoccer well hopefully mtgox will attempt to actully fix it
12:03 money whats that??
12:03 starsoccer otherwise looks like we will be waiting weeks for mtgox
12:04 Ademan gox has been dead to me for almost a year now
12:04 money so the bitcoin devs have to fix that and then they will open bitcoin withdrawals?
12:04 wallet42 money: defn not
12:04 sipa money: hell no
12:04 gmaxwell they're going to be wating a long time for that.
12:04 money Also does this mean that the whole network is currently vulnerable???
12:04 Apocalyptic money, the bitcoin dev don't have to do shit
12:04 money well he said it.
12:04 wallet42 money: thay have to fix THEIR implementation
12:04 gmaxwell money: you were trolling hard in the gox channels earler, please don't bring it here.
12:04 Apocalyptic so it must be true ?
12:04 puzl but they can track the mutation through the blockchain, right?
12:05 money NO I quoting what the guy said from the mtgox.com site.
12:05 maaku puzl: yes
12:05 puzl so it should be easy enough for them to fix the problem on their end?
12:05 maaku puzl: yes again, for some definition of "easy"
12:05 wallet42 puzl: well i dont think its a cakewalk
12:05 gwb3 i think it would be helpful if other exchanges started coming out in defense of bitcoin
12:05 sipa there are 3 different problems here: malleability as a whole (which we are slowly working towards fixing, but will take years), 2) wallets (like mtgox's) which don't deal well with malleability and 3) other services that rely on a constant txid to track transactions that do not confirm
12:05 money I'm trying to technically understand this. just a second gmaxwell. He said he took this bug with the bitcoin core developers and will only open bitcoin withdrawals once they fix this bug.
12:05 sipa it is for 3) that they want a standard solution, and we're working on proposing one
12:06 sipa however, this is NOT something that will impact the protocol or clients
12:06 gwb3 last week (and even to a degree before that) mt. gox really got dragged through the mud
12:06 puzl well, for a definition of easy being "all the information is present already" i.e. no protocol change required
12:06 gwb3 their brand certainly tarnished to a degree
12:06 gmaxwell sipa: well kind of a 4th too, did you see my point about inputs? gox wouldn't have been saved from 'potential' losses by just the elimination of mallibility because they respent without doublespending.
12:06 maaku money: that statement was factually incorrect as we have described to you above. please read the scrollback
12:06 gwb3 this press release is basically saying, "hey, it's not us, it's bitcoin - good thing we picked up on this, others could be affected"
12:07 money He also states that users can say they didn't get a tx, by manipulating the bitcoinchain thing. he says that anyone can do it right now, and claim that they didn't get a pay.
12:07 money ok maaku , I just copy pasted from his site
12:07 money https://www.mtgox.com/press_release_20140210.html
12:07 Hawkix can someone SCAN the blockchain to detect all mutated transactions and post stats about the misuse?
12:07 maaku Hawkix: no
12:07 wallet42 you cannot FIX the "doublespending" problem by respending
12:07 Ademan Hawkix: not from the blockchain alone
12:08 maaku because only one of the transactions made it on the chain
12:08 Apocalyptic Hawkix, that would be their job
12:08 gwb3 exchanges are integral to the sustainability of bitcoin, also confidence in exchanges are integral to continued adoption by merchants and other retailers
12:08 Hawkix I thought that to change transaction id, one must add some padding to the script, right?
12:08 Ademan Hawkix: I suppose you could try to listen to all transactions on the network, and see if any mutants of transactions you've seen make it into the blockchain
12:08 gwb3 more retailers are adopting bitcoin each day
12:08 maaku Hawkix: but if by "someone" you mean mtgox, then yes, they have the necessary info to do that
12:08 wallet42 Hawikix: https://en.bitcoin.it/wiki/Transaction_Malleability
12:09 maaku or they should if they kept logs of their transactions
12:09 Ademan Hawkix: Both mutant and original transaction are valid, it's not really possible to tell which is the "original"
12:09 Apocalyptic (this is clearly off-topic here=
12:09 gwb3 does anyone in here know someone in the community who i can reach out to about marketing strategy coming off this press release?
12:10 gwb3 honestly if other exchanges start coming out against what mt. gox just said i think it will help
12:10 Ademan Apocalyptic: the nature of the blockchain and protocol are offtopic here?
12:10 Apocalyptic Ademan, nope, the talk about gox
12:10 money So what mtgox said is wrong? then? --> this cannot happen: In the meantime, exchanges and wallet services - and any service sending coins directly to third parties - should be extremely careful with anyone claiming their transaction did not go through.
12:10 Ademan Apocalyptic: ah, sorry
12:10 randy-waterhouse gwb3: most retailers are using BitPay ... talk to them?
12:10 gwb3 randy-waterhouse: ok
12:11 maaku money: that part is correct. services should not be checking if transactiosn went through by txid
12:11 Apocalyptic ^
12:11 money I see.
12:11 Plarkplark maaku: cant they just check the receiving address, and wait for 3 or 6 confirms?
12:12 maaku no, because anybody could send coins to the receiving address
12:12 Apocalyptic Plarkplark, it's a bit more complicated than that, the guy can claim he's expecting another tx with the same amount
12:12 Apocalyptic you have to watch for specific inputs there
12:12 Apocalyptic which can be done
12:12 Plarkplark Ah. There we go. thanks
12:12 Plarkplark How do other exchanges handle that?
12:12 maaku but they can extract the essential skeleton of the transactions (inputs minus the script sigs + outputs) and watch for that
12:12 Plarkplark Or is that too complicated?
12:13 Plarkplark if you pull apart he receiving address and check if the balance created contains your inputs, you are ok, right?
12:13 Plarkplark (if it is confirmed of course)
12:14 money so what's the way to comfirm that the tx went through ok then?
12:14 Ademan money: if there's a transaction in the block chain which has the inputs and outputs expected
12:14 randy-waterhouse I think a lot of folks have lost sight of the fact that bitcoin is a much better settlement/clearing system than it is a payment system
12:15 randy-waterhouse a la jgarzik
12:15 money ok
12:15 money and a software and check for that easy? or does it neeed manual intervention?
12:16 Plarkplark so Gox was not doing things best-practice. and they got hurt.
12:16 Ademan randy-waterhouse: I think the distinction is lost on me, then again it's really late/early... what do you mean?
12:16 money but it's annoying to see them blaming others for their stuff. and confusing too.
12:17 Ademan money: Software to check the blockchain for your transaction is easy enough.
12:17 money ok