New Caledonia is one of the few places in the South Pacific where camping is widely understood and accepted. French soldiers on leave from Nouméa do it all the time, and French locals consider it a legitimate way to spend a weekend.
There are many organized campgrounds on Grande Terre (but none in Nouméa). On the west coast, the municipal campgrounds are usually far off the main highway and not accessible by public transport, while those at gîtes on the east coast are often right on bus lines. On the outer islands, you also camp at the gîtes.
Almost anyone you ask will readily give you permission to pitch your tent, but do ask. Otherwise you might be violating custom and could create a needless problem.
If camping for free on a public beach, take care of your gear.