The country will spend 19.5 billion yen (£139m) on the computer, which should make 130 quadrillion calculations per second.
That is 130 petaflops in scientific parlance.
It is part of a government policy to keep up with South Korea and China, currently home to the world’s best-performing machine.
The country’s engineers will be tasked with building the machine as early as next year, sources told Reuters.
At 130 petaflops, Japan’s computer would be ahead of China’s Sunway Taihulight, which is capable of 93 petaflops.
“As far as we know, there is nothing out there that is as fast,” said Satoshi Sekiguchi, from Japan’s National Institute of Advanced Industrial Science and Technology, where the computer will be built.
Prime Minister Shinzo Abe wants companies and politicians to work more closely so Japan can win in areas such as artificial intelligence.
The computer should advance “deep learning” technology that works off algorithms which mimic the human brain’s neural pathways.