The U.S. has no intention of withdrawing troops from South Korea as part of any agreement with North Korea, Defense Secretary Jim Mattis said.

“I really don’t know where this stuff comes from,” Mattis told reporters. “Once someone has made it up, apparently it gets a life of its own.” Mattis’ remarks came as the U.S. and North Korea are preparing for a possible June 12 summit in Singapore.

The U.S. has more than 23,000 troops along the Korean Peninsula and expects to dedicate its new headquarters building at Camp Humphreys, near the city of Pyeongtaek, at the end of June. The Eighth Army headquarters has already moved.

The full relocation from Yongsan Garrison in Seoul to Camp Humphreys won’t be completed until 2021.