There’s no sunlight four kilometres below the waves but there is light. It comes from a sea cucumber that emits a faint glow from its sticky skin, attracting fish and crabs that try to take bites out of its side.

The skin is both a lure and a trap, marking incautious predators with a sticky glowing dot, an “eat me” sign to any passing larger predators. It comes from the anglerfish, winking their lures on and off to attract prey in the inky darkness, and the sea stars, which emit bursts of light to ward off predators.

In the absolute darkness there are busts of colour, the blues and greens of an alien landscape, where as many as 80 per cent of organisms have some capacity for bioluminescence. Until recently, no scientists had explored these depths on the Australian coastline.

On Friday, the CSIRO research vessel Investigator docked in Brisbane at the end of a 31-day voyage from Tasmania to the Coral Sea, mapping the sea floor, collecting specimens and trawling for rubbish the whole way. Its hold was a wunderkammer of gelatinous fish and spiny crabs, scarlet sea-spiders, nightmarish cookie cutter sharks and bioluminescent invertebrates.

One of the passengers, the fleshy, newly discovered faceless fish, has already made international headlines. More than 1,000 different species were collected, at least a third of which were new to science.

Of 200 organisms tested in a dark lab by Jerome Mallefet, an international bioluminescent expert and researcher at the National Science Foundation in Belgium, just over half showed some kind of bioluminescent response. That’s a conservative estimate, Mallefet said.

The 4km journey to the surface in a metal box was not a kind one and many specimens died before reaching the surface. While some exhibited a posthumous response, Mallefet said, it’s unknown how many others might have glowed when they were alive.

“Here what I have seen is the response they give me on the surface,” Mallefet said. “But we really have no idea of the functioning of bioluminescence down there. In these really deep waters we don’t know nearly anything about them.”

Some creatures were collected by the sampling box on the way up from the sea floor. Among them was a cookie cutter shark, a species that can live anywhere from 200m to 3km below the surface but is often found in an area known as the twilight zone, 1km below, the deepest point that sunlight can reach.

The cookie cutter shark also glows. “If you are in the twilight zone, you are able to be seen from below as a shadow,” Mallefet said. “But if you emit light from your belly at the same colour as the light above you, you become invisible.”

The RV Investigator left Hobart on 16 May and travelled up the edge of the continental shelf. The ship is equipped with state-of-the-art sonar equipment, which captured a 12km wide swathe of the ocean floor, providing the first accurate picture of the sea floor off the Australian coastline.

The mapping technology is never turned off while the ship is in motion and research crews are instructed to try and avoid sailing over the same patch of ocean twice, to add to the ocean floor data collected by the CSIRO.

“On the sea floor there are mountains and valleys and crevasses and canyons — it’s really rugged down there,” Dr Tim O’Hara said. O’Hara is the senior curator of marine invertebrates at Museums Victoria and was the chief scientist on the expedition, heading up a team of 27 scientists, 13 technicians and 20 crew from seven countries. He said previous mapping of the sea floor along the Australian coast assumed it was more featureless.

The rugged terrain made for difficult sampling. A metal box, known as a sled, was lowered and dragged along the sea floor before being brought up, collecting other creatures and often rubbish on the way. Related: Epic Antarctic voyage maps sea floor to predict ocean rise as glacier the size of California melts

“I don’t think there has been a single sled load that we have done that hasn’t brought up a bit of rubbish,” O’Hara said. The most exciting finds, O’Hara said, had been the wide array of deep-sea fish that look like Pixar concept art. There were the lizard and dragon fish, with an “amazing variety of teeth”, the faceless fish that O’Hara said had “such an enigmatic face” despite eponymously not having a face at all, what appeared to be a new species of blobfish, recovered off the coast of Tasmania, a bright red coffinfish and a new species of anglerfish. Many of them were decidedly blobby.

“In the deep sea it doesn’t pay to have a lot of muscle because there’s so little food down there,” O’Hara said. “It is much better to have a jellylike body and just wait on the sea floor for prey.”

The expedition also collected a shortarse feelerfish, a species discovered in New Zealand and named for the unusual and potentially unwise proximity of its anus to its mouth.

It was only the second specimen found in Australian waters. On a whale skull they found a number of zombie worms, mouthless fleshy tubes that hang around until bacteria has broken down the bone marrow and suck up the resulting soup. “It’s an extraordinary lifestyle,” O’Hara said.

Taxonomists will now sought through the specimens to describe and name any new species. A selection will go on display at a temporary exhibit at Melbourne Museum later this year.