We paid $634 million for the Obamacare sites and all we got was this lousy 404

For all of you that didn&#x27;t know, this project was never put out for bid with an RFP. It was sole-sourced to CGI Federal without any competitive bids.<p>&quot;Federal health officials have not yet explained why CGI was given the contract or why it was awarded on a sole-source basis.&quot;<p><a href="http://www.washingtonpost.com/business/technology/some-say-health-care-sites-problems-highlight-flawed-federal-it-policies/2013/10/09/d558da42-30fe-11e3-8627-c5d7de0a046b_story_1.html" rel="nofollow">http:&#x2F;&#x2F;www.washingtonpost.com&#x2F;business&#x2F;technology&#x2F;some-say-h...</a><p>[Edit to add source]

If it really did cost over half a billion dollars for only 50k parallel connections, that is outrageous.<p>It is hard to read&#x2F;understand that spending breakdown, not sure what each 20-50 million dollar allotment is exactly for:<p><a href="http://usaspending.gov/explore?tab=By+Prime+Awardee&amp;fiscal_year=all&amp;idvpiid=HHSM500200700015I&amp;typeofview=detailsummary" rel="nofollow">http:&#x2F;&#x2F;usaspending.gov&#x2F;explore?tab=By+Prime+Awardee&amp;fiscal_y...</a><p>Given how we can see the problem with this as developers, imagine what we don&#x27;t understand about the 10 to 1 or even 100 to 1 overspending on the military and &quot;defense&quot; budget. Just imagine the TSA and &quot;Homeland Security&quot; waste.

Reread the article, this is complete nonsense. The $634M number is 6 years of contracts, starting in 2008 (Obamacare was signed into law on March 23, 2010), which doesn&#x27;t align with his claim that CGI Federal won the contract in 2011.<p>The solicitation number he uses to filter the USASpending.gov site is also tied to a larger PECOS contract, and not to Healthcare.gov like he claims.<p>We&#x27;re being had guys. This is poor reporting at best, or deliberate obfuscation at worst.

This is what frustrates me about many of our non-USA HN&#x27;ers here: every time the state sponsored health care issue comes up, the non-USA folks can&#x27;t wait to talk about how their country has it, or how it costs this little to get coverage for this or that problem in their country, and then they wonder why &quot;you Americans&quot; put up with the system you have currently in place (self-insured, no state plans - pre-Obamacare). I&#x27;ve read my share of comments (often from European HN&#x27;ers) that mock USA citizens&#x27; mentality with regard to state sponsored health care here.<p>Well here you go - this is the perfect example to share with you as to why most Americans didn&#x27;t&#x2F;don&#x27;t want &quot;Obamacare&quot;. We, the voters, have no confidence that the current system+administration+Congress (or previous 10 of each!) could&#x27;ve created a system for &quot;health insurance for all&quot; that worked and was efficient. It&#x27;s not anything against Obama - it&#x27;s that we&#x27;ve seen administration after administration try to implement some big, sweeping group&#x2F;plan for 20+ years and every one of them has turned into an inefficient holy hell of a mess. The most recent example of a major #$%&amp;-up is Homeland Security (which I think is the last major agency created).<p>If the American public believed that the current government was capable of delivering state sponsored health care in an efficient way, every Democrat and Republican in this country would&#x27;ve voted for it. So don&#x27;t think of &quot;those who are against Obamacare&quot; as anti-Obama, but rather anti-inefficiency (or anti-bigger government).

Hi, I actually work for CGI Federal on a closely related website to healthcare.gov. There&#x27;s some inaccurate speculation going on in this thread that I want to correct. Employees at CGI Federal are not payed hourly. Also, a lot of people have mentioned cronyism, which I think is baseless. CGI Federal already had contracts for Medicare.gov and CMS.gov, so when CMS had to build healthcare.gov, CGI was an obvious choice.<p>edit: Also, the projects I work on are Agile and I get to use other technologies besides .Net framework (e.g. node.js and backbone.js)

As has been noted on HN at the time, the front-facing part of the site was released as open-source 3 months ago:<p><a href="https://github.com/CMSgov/healthcare.gov" rel="nofollow">https:&#x2F;&#x2F;github.com&#x2F;CMSgov&#x2F;healthcare.gov</a><p>You can deploy it like any other Jekyll site. The code and content has changed since then but I imagine it still has the same static front facing architecture, much like the Obama campaign fundraising site, which famously raised $250M using Jekyll static pages: <a href="http://kylerush.net/blog/meet-the-obama-campaigns-250-million-fundraising-platform/" rel="nofollow">http:&#x2F;&#x2F;kylerush.net&#x2F;blog&#x2F;meet-the-obama-campaigns-250-millio...</a><p>So the number of raw visitors may not be the most relevant number, as many of them may have hit the front page and left, or never got around to the signup part. But what exactly was the technology in the back-end stack?

Political organizations exist for political reasons.<p>So if you get elected to &quot;do something&quot; about orphans, you&#x27;ll create an organization to do something. The goal of this organization is political: appear to be making progress on the orphan problem. At the very least, do not appear in the news as an example of government waste.<p>The current website problem is a political failure -- it looks bad. But that&#x27;s just a short term consideration. The long-term bet is that over the next decades, the ACA will bring great political benefit to the political party that supported it, no matter what other things it does.<p>So when we evaluate projects created by political organizations for political reasons, the success criteria is much different than commercial or non-profit projects. I don&#x27;t think this is a failure. Maybe a bump in the road, but it&#x27;s nothing that won&#x27;t work itself out over the next year or so. (And be long forgotten)<p>Remember, a lot of government contractors made money building these sites. A lot of people had jobs. A lot of committees and functionaries are able to add this to their list of good they&#x27;ve done in the world. Not being able to actually use the site for a while is a small pittance compared to the real, measured benefit the sites have created. So far. If it drags on for a long time, the political math could switch around the other way, but I doubt it.

National Public Radio had a good report on 8 October 2013, &quot;Health Exchange Tech Problems Point To A Thornier Issue,&quot;<p><a href="http://www.npr.org/blogs/alltechconsidered/2013/10/08/230424841/health-exchange-tech-problems-point-to-a-thornier-issue" rel="nofollow">http:&#x2F;&#x2F;www.npr.org&#x2F;blogs&#x2F;alltechconsidered&#x2F;2013&#x2F;10&#x2F;08&#x2F;230424...</a><p>that discusses the broader issue of United States federal government contracting for information technology services. Some reforms are suggested in that story that would help more competent startup companies compete against the established federal contractors that win most of the big contracts. The specialized skill that the incumbents bring to the contracting process is not specialized skill in data-processing or programming for federal agencies, but rather specialized skill in navigating the federal bureaucracy for bidding on federal contracts.

There are a lot of interesting things to say about a $634M website that didn’t work when it should have, but what pops out at me here is the description of “poorly written code.”<p>This article even links directly to some of the code, which A) looks fine? and B) is like “Yeah, the problem wasn’t with procurement regulations or clear requirements or public-private cronyism or managerial competence, it was that damn person who told the computer what to do and the computer didn’t do the right thing.”

I lost my group health plan, which I had for fifteen years, recently so I has experienced the web site first hand :-)<p>My latest complaint: I logged in many times last week to fill in information, a bit at a time before I lost access.<p>Then starting Sunday morning I could no longer login. On the phone last night I was told that they had wiped all stored passwords and to follow the &quot;I don&#x27;t remember my password&quot; link. Couldn&#x27;t they have posted that as a huge banner message and let people know? I had four days of frustration trying to login.<p>Then, when I got the reset password link, and followed it, one of the profile verification questions they asked for resetting my password WAS A QUESTION THAT I HAD NOT BEEN ASKED so I had no way to answer it. I wrote down my profile questions so I am fairly sure of this. wTF

I&#x27;ll never understand how it&#x27;s possible that government web sites cost so much, yet usually look like shit, work like shit and are easy to hack (at least in my country). I always get the feeling that companies that win the contracts are somehow related to people responsible for the decision, it&#x27;s like &quot;Hey, it&#x27;s not our money, so let&#x27;s milk the budget as much as possible!&quot;.

This seems a little confusing to me. I&#x27;m using the site linked in one of the previous comments:<p><a href="http://usaspending.gov/explore?tab=By+Prime+Awardee&amp;fiscal_year=all&amp;idvpiid=HHSM500200700015I&amp;typeofview=detailsummary" rel="nofollow">http:&#x2F;&#x2F;usaspending.gov&#x2F;explore?tab=By+Prime+Awardee&amp;fiscal_y...</a><p>This shows the $634M headline number as the amount of contracts paid to CGI Federal over a ~6 year period. It is filtered based on solicitation number HHSM500200700015I.<p>A little googling returns the following page: <a href="https://www.fbo.gov/index?s=opportunity&amp;mode=form&amp;id=f1522d0262c63d01b34fab07c6ed5634&amp;tab=core&amp;_cview=0" rel="nofollow">https:&#x2F;&#x2F;www.fbo.gov&#x2F;index?s=opportunity&amp;mode=form&amp;id=f1522d0...</a><p><pre><code> &quot;The Centers for Medicare &amp; Medicaid Services (CMS) intends to modify the PECOS contract to support the American Recovery and Reinvestment Act of 2009 (ARRA) requiremnts for the Development, Maintenance and Enhancements of HITECH Registration, Attestation and Inquiry Functionalities. This work is already on the contract, the modification will incorporate costs for the option years.&quot; </code></pre> So there&#x27;s not really a lot of detail on what the HHSM500200700015I solicitation actually entails, and I haven&#x27;t found a clear description everywhere else. It looks like it could cover a pretty broad set of work. For example, the ACA was signed in March 2010, but there&#x27;s contracts tied to this solicitation number that go back to 2008-2009.<p>A few of the news sites linked below imply that the $634M covers the total number of Medicare and PECOS contracts awarded by HHS to CGI, and that the ACA website cost only $93M, which makes it seem cheap compared to the private sites listed in the Digital Trends article:<p><a href="http://washingtonexaminer.com/canadian-firm-hired-to-build-troubled-obamacare-exchanges/article/2536805" rel="nofollow">http:&#x2F;&#x2F;washingtonexaminer.com&#x2F;canadian-firm-hired-to-build-t...</a> <a href="http://www.foxnews.com/tech/2013/10/09/3-million-obamacare-website-may-face-months-glitches-experts-warn/" rel="nofollow">http:&#x2F;&#x2F;www.foxnews.com&#x2F;tech&#x2F;2013&#x2F;10&#x2F;09&#x2F;3-million-obamacare-w...</a><p>Before we all explode in outrage, are we sure that we are reading the details of this correctly?

A colleague of mine used to be some kind of manager&#x2F;director at that company (in another country though). He once described his former job as &quot;milking the government&quot;.<p>They hire the lowest quality engineers (e.g. the cheapest), and then bill them outlandish $&#x2F;hour to work for government agencies. The project plans are packed with useless stuff, and are designed to exceed initial estimations. They get paid by the hour...

Things not considered by this and other articles that I would urge readers to consider: 1) RFPs are often written poorly by non-technical people, with requirements that are not accurate at the time. These requirements then change a lot to reflect reality which results in a lot of wasted effort and redirection. (This is probably also the case in large Enterprise implementations)<p>2) Compliance with government regulations costs money. Lots of money. This results in a lot more overhead. It also results in a lot more time to get people up to speed, on site, and going. This is why government contractors keep winning bids - compliance costs are huge barriers to entry.<p>3) Systems you need to integrate with in government (especially legacy systems) can be a complete pain in the butt. It&#x27;s more likely you&#x27;re integrating with some FORTRAN green screen than a nice JSON API. This makes large scale systems integration hard.<p>That said...the app is still very broken and there is obviously a failure here. Failure to test properly (otherwise poorly written tests), failure to open to competitive bids judging from another comment in this thread, and many other issues.<p>There is a LOT to be done to improve IT acquisition in government, and many things should have gone right that went wrong for the money spent (Figures I saw were more like $138 Million in other publications), but readers should please consider the organizational barriers and difficulties that exist and then factor it into the cost. It doesn&#x27;t take the sting away, but it does lessen it a bit.

Hmmm. I just created an account. The site loaded quickly. Took maybe 2 minutes to create an account. The password rules were weird, but I don&#x27;t see what all the fuss is about. It seems to me that scaling a site from 0 users to millions in a couple of weeks is hard.<p>If it&#x27;s working now, which is seems to be, what&#x27;s the problem? (OK, I&#x27;ll concede that $634M is totally and completely insane. But I&#x27;m not convinced by the rest of the &quot;bad code&quot; complaints.)

Scaling on the web is basically a solved issue for sites with such limited functionality. Even the CIA is using AWS, so I can&#x27;t believe the regulatory and security hurdles were prohibitive.<p>To spend this kind of money on sites with these kinds of results is just disgraceful:<p><i>&quot;The site is so busted that, as of a couple days ago, the number of people that successfully purchased healthcare through it was in the &#x27;single digits,&#x27; according to the Washington Post.&quot;</i><p><a href="http://www.cbsnews.com/8301-505269_162-57606633/obamacare-website-looks-like-nobody-tested-it-programmer-says/" rel="nofollow">http:&#x2F;&#x2F;www.cbsnews.com&#x2F;8301-505269_162-57606633&#x2F;obamacare-we...</a>

Eisenhower warned about the military industrial complex. This is just the latest manifestation of the government-industrial complex. I&#x27;d be willing to bet that CGI Federal (the contractor) has some very well connected cronies to have captured this boondoggle.

&gt; the factors that play into which companies receive government contracts, a process called “procurement,” are fundamentally broken<p>This.

Imagine what this community could build with more than half a billion dollars!

This is absolutely nothing compared to what they waste on technology in DOD...

I have worked with IT consulting for the government (in Brazil), and if the software needs of the american government are similar to the Brazilian ones, you can&#x27;t compare twitter, instagram, etc. with this! They are completely different kinds of software, governmental softwares have tons of integration with old COBOL systems that runs millions of sensible data, and tons of requirements, really, pages and pages of different use case scenarios. Governmental software require more man-hours than successful start ups, even when they work and succeed as a project. The &quot;MVP&quot; of a governmental system is huge. Now, don&#x27;t get me wrong. the price is absurd, this example is clearly a total failure. It&#x27;s just that this article compare apples to oranges.

The sad part about this is, from what I can tell, there&#x27;s not going to be any fixing this any time soon. The company who wrote the code doesn&#x27;t know what they&#x27;re doing, and nobody else is ever going to be able to figure it out NOW. Sad state of affairs.

Good thing we are giving the federal government control over the healthcare system ( and positioning them to take even more control as we move towards single payer ). I am sure they won&#x27;t screw it up like they have the the roll out of this small portion of the system.<p>Also I am really looking forwards to the day I can&#x27;t get my kid seen by a doctor because there was a budget fight and the government shut down.

It definitely seems like the amount of money the government spends on projects like this starts out at something outrageous no matter how complex.

They should have opened the bidding process to Elance, they could have found someone to do the back-end for like 5K. ;)<p>ETA:<p>&gt; And when things still go wrong, they simply throw “more money at the same people who caused the problem to fix the problem.&quot;<p>For that much money, I could be a terrible web developer.

Haha, yeah, I spend hours across two days on those government health exchange sites (the US one and NY one) just trying to get a quote. Kept erroring out and forgetting my history and other problems.<p>Meanwhile Freelancer&#x27;s Union site was great: <a href="http://freelancersunion.org/" rel="nofollow">http:&#x2F;&#x2F;freelancersunion.org&#x2F;</a><p>I got a quote easily before I had to do any quizzes. The gov ones were not like that. There were a few documents that had to be emailed in for verification, but they took care of that promptly. There was one bug related to zip+4 not being supported and not being changeable after entering, but they answered the phone immediately and fixed it in a minute.

As someone who has worked on the front end of a couple of .govs, dealing with 1 mid level IT manager is enough. Having them all &quot;collaborate&quot; is just more chaos. The culture of the agency (DOD vs HHS) can also be difficult to work with.

As more and more legislation starts to be implemented via online apps, it seems almost inevitable that hacktivists will eventually attempt a filibuster by DDoS. Given the turbulent launch of the new federal healthcare exchanges, my confidence that the government would be able to handle such an attack is rapidly diminishing.<p>I wonder how the political climate might shift if the threat of a de facto &#x27;Annonymous veto&#x27; became something that policy makers actually had to worry about. On the up shot, it might finally force law makers to gain a responsible level of internet literacy...

Ok here&#x27; some other contractor news, Serco in the UK under fraud investigation home of their corporate offices, one of the exchange contingency contracts given at the last minute as well as Equifax...<p><a href="http://ducknetweb.blogspot.com/2013/07/government-gives-contract-to-equifax-to.html" rel="nofollow">http:&#x2F;&#x2F;ducknetweb.blogspot.com&#x2F;2013&#x2F;07&#x2F;government-gives-cont...</a><p><a href="http://ducknetweb.blogspot.com/2013/07/obama-administration-doles-out-yet-one.html" rel="nofollow">http:&#x2F;&#x2F;ducknetweb.blogspot.com&#x2F;2013&#x2F;07&#x2F;obama-administration-...</a>

I told <a href="https://www.healthcare.gov/" rel="nofollow">https:&#x2F;&#x2F;www.healthcare.gov&#x2F;</a> that I want to apply and live in New York. It immediately gave me a link to <a href="http://nystateofhealth.ny.gov/" rel="nofollow">http:&#x2F;&#x2F;nystateofhealth.ny.gov&#x2F;</a> . I clicked the link, and it says:<p>&gt; Access Denied &gt; You don&#x27;t have permission to access &quot;<a href="http://nystateofhealth.ny.gov/&quot;" rel="nofollow">http:&#x2F;&#x2F;nystateofhealth.ny.gov&#x2F;&quot;</a> on this server.<p>Why would the top level of such a site have permissions at all?

this is an absurd waste of taxpayer money. we shouldn&#x27;t let the government and their contractors get away with this sort of robbery. it is no wonder we are so far in debt as a nation.

I&#x27;m less concerned about the short term scalability issues as I am about the long term security and privacy issues. The code quality doesn&#x27;t give me a lot of hope.

has anyone here actually attempted to use the site, or just reading about how it fails? The only commenter I&#x27;ve read was using IE.<p>I just signed up and had no problems.

A law fining Americans for not buying health insurance is not a &quot;tax&quot;, it&#x27;s a fine. It&#x27;s a civil punishment. Double-speak. The sky is still blue even if the Supreme Court rules its green. The individual mandate is a civil penalty just like a parking ticket. The supreme court is a joke and the commerce clause crap has gone beyond the absurd.

Interestingly CGI is a Canadian company...

Why did the government need to build any website? Why couldn&#x27;t they just release data, and pass rules that health companies release data? Many websites would have offered to help bring the data to the users and the Government could have linked to some officially recognized ones.

I don&#x27;t see the problem - just look at the site.<p>It has a nice graphic that takes up two thirds of the page while giving no significant information.<p>What could possibly be wrong with that? That&#x27;s what most modern landing pages are doing, after all.

So this is what $634mm gets you?<p><a href="https://www.healthcare.gov/marketplace/global/en_US/registration.js" rel="nofollow">https:&#x2F;&#x2F;www.healthcare.gov&#x2F;marketplace&#x2F;global&#x2F;en_US&#x2F;registra...</a>

Out of curiosity, what is the proportion of non-governmental, large-scale site launches that have gone well?<p>Seems like I can remember several very public commercial blow-ups.

Site just crashed my IE8.<p>Generally I have had nicer experience with most sites developed under Obama administration like wh.gov and such.

I can&#x27;t sign up without js enabled. Should this site especially adhere to strict accessibility standards?

Should have got Harper Reed and the Obama campaign tech team to knock it up in Rails on AWS.

Who is in-charge for healthcare.org? What company?

Maybe it wasn&#x27;t ever intended to work.

I&#x27;m shocked that people seem to think 3 years and $634m is a lot for a project of this scale.<p>Are you all offering to design and build it 6 months in PHP for $50k ?