The IJF launches ‘Open By Default,’ a new database of internal government documents never before made public

Today, the Investigative Journalism Foundation launches its newest database, “Open By Default,” a repository of internal government records released under federal access legislation.

Open By Default shares more than 20,000 (and growing) completed federal Access to Information (ATI) requests as part of the IJF’s mission to increase government transparency.

Open By Default aims to “help people better understand how their government works behind the scenes,” said Zane Schwartz, CEO and editor-in-chief of the IJF.

The brainchild of Thompson Rivers University assistant law professor Matt Malone, Open By Default was started over two years ago after he discovered both the utility and difficulty of navigating the access to information system for his own academic research.

“The access to information system in Canada is widely criticized for being inefficient, outdated, and overly secretive,” said Malone. “One thing I really wanted to do was fix that and make sure that we are putting these records into the public domain so that journalists, researchers, businesses and citizens can have instant and unhindered access to them.”

The name, Open By Default, comes from a promise made by Prime Minister Justin Trudeau in 2015 shortly after being elected. “Government and its information must be open by default,” Trudeau had written at the time.

“This was a grand promise of transparency from this government which, of course, was never met,” said Malone. 

Until today’s launch, anyone wanting to view a previously-fulfilled ATI request had to jump through several hoops. The government has made only brief summaries of previous requests available to the public. After identifying a desired ATI request, you must fill out a form (known as an “informal request”) to ask the department to send you a copy.

Then you wait. And, unlike initial ATI requests, there’s no legislated time frame within which the government must respond. Many users can attest to often waiting many months — or longer — to get sent the documents requested.

But no longer. Now, with a few clicks, you can access over 2.6 million pages of documents released under the ATI program, searching and filtering by keywords, date received and organization.

To create the initial prototype of Open By Default, Malone partnered with Adam Soames, a master’s student in environmental science at Thompson Rivers University.

Soames said when Malone first approached him about the project, he was excited about Malone’s vision and eager to take on the programming challenges.

The technical challenges proved to be extensive. Starting in June 2023, Soames worked with Malone to build a prototype website and set up an automated system for requesting completed ATI requests from government departments.

The initial work was tedious, involving tying each document to the ATI request summaries provided by the government, sorting through manila envelopes and hours of uploading countless pages — and even CDs — mailed to them by departments.

Despite having overcome several technical hurdles, Malone and Soames then faced a bigger challenge: backlash from the government.

“The government has been very hostile to this project with very few exceptions. There are a couple folks in the public service who have been absolutely stellar in supporting the project. But on the elected side, there’s been a lot of hostility,” said Malone.

Soames said Malone felt extremely discouraged and they had to gather the courage to keep working on the database. 

“There are a lot of very powerful people out there who probably don’t want this project to succeed. And if we’re going to be successful in fighting them off, we’re going to need some help,” added Soames.

Enter the IJF. With the IJF’s technical resources, Malone and Soames said they were thrilled to partner with the non-profit news outlet.

Newly partnered with the IJF, Soames worked with the IJF’s technical team, primarily web developer Daniel Nass and data analyst Sam Park, to expand the scope of the repository and add key features, such as the ability to search through not only summaries of the ATI requests but also within the files themselves.

“The biggest challenge was the sheer volume of the material we’re working with,” said Nass, adding that the documents “tend to be very long, on average, more than 100 pages.”

To handle the hundreds of gigabytes of documents and add in-text searching using optical character recognition technology, the IJF’s technical team relied on the non-profit MuckRock Foundation’s DocumentCloud service.

“Being part of this team has really been an honour,” said Malone. “The commitment, the ingenuity, and the perseverance of the core technical team that has built the final project is really incredible.”

Journalist Dean Beeby is well-known in the industry for his extensive and pioneering use of the ATI system for public interest reporting.

Beeby has been filing ATI requests since the 1980s. He said he’s noticed a visible decline in the system since the early 2000s, mainly through an increasing reliance on the defence of “cabinet secrecy” to redact information and delays that grow longer and longer by the year.

“Governments have, over the years, grown more savvy about what they can get away with. And we’re just at a point where no government really wants the trouble of fixing it, because fixing it is going to mean more exposure, more documents and potential embarrassment,” said Beeby.

On top of the increased public accessibility of government documents provided by Open By Default, Beeby said one of the main benefits of the database is its role as an archive. 

Government departments are only required to store access to information response packages for two years, after which they can choose to dispose of them. While some departments hold onto them longer, others dispose of them immediately.

“The documents themselves often just are lost and Open By Default has stepped into the breach here and is capturing them. And that is a huge benefit to the country and to journalism and to anyone else who uses access [to information],” said Beeby.

Open By Default builds upon a history of people trying to fill the gaps created by the government, including the Globe and Mail’s Secret Canada project, the Sunlight Project from the Investigative Journalism Bureau and University of Toronto Libraries, professor Tim Sayle’s Canada Declassified, and personal efforts from researcher Ken Rubin and developer Laurent Bastien Corbeil.

But Beeby said the government itself should step up and do the job that these organizations and individuals have taken on.

While the public continues to wait for long-promised reforms, members of the Open By Default team said they hope that all Canadians can find value in Open By Default.

“Our mission is to strengthen Canadian democracy. And the Open By Default database is a fantastic complement to our other databases,” said Schwartz.

“The thing that makes this database great is that it’s so open-ended. There’s something in there for everyone no matter what your beat is or your subject area or your profession,” said Nass. “I’m excited to see the public just jump in there and start exploring and see what they discover.”

---

Try out our newest database yourself! Visit theijf.org/open-by-default to see what you can uncover.