Firstly, talk like a local! Start calling it tramping and let's never speak of hiking again.
Entering the country
Be aware that New Zealand has strict regulations around biosecurity. Declare any outdoors gear and food on entry so that it can be checked. If you're not sure, just declare it. Make sure your boots, walking poles, tent (including poles and pegs) are thoroughly cleaned. Read more: Biosecurity.
Weather
New Zealand weather is changeable and hard to predict. Prepare for heavy rain, and snow in the mountains at any time of year. Seasons are reversed compared with the northern hemisphere, so summer is December-February and winter is June-August. The north of the North Island is warmest, while the south of the South Island is coolest. Temperatures are typically 5-15 degrees Celsius in winter and 10-25 degrees Celsius in summer. Mid-summer tramping can be hot, even in the mountains. Autumn and Spring are probably the most pleasant seasons for tramping.
Animals
Lacking native land mammals (aside from a couple of bats), New Zealand became a land of birds. Strange creatures like kiwi evolved to fill the niches left on the ground. Today, introduced predators (rats and mice, mustelids, possums, cats, dogs, and even hedgehogs) pose a grave and ongoing risk to native birds. Extensive trapping and poisoning campaigns are run, while many offshore islands are used as sanctuaries. Several of these islands are open to casual visitors. New Zealand has no dangerous animals aside from sea lions on the coast.
Geography
New Zealand is a chain of three main islands about the size of the United Kingdom or Japan. From north to south, these islands are known as the North Island, the South Island and Stewart Island or Rakiura. The national population is just over 4 million people.
North Island
The North Island is a rolling landscape of forest and farmland. Many North Island tracks explore forest without breaking out into the tops. Strikingly, the North Island is scattered with volcanoes, from the smaller cones around Auckland to the mountains of Taranaki, Ruapehu, Tongariro, and Nguaruhoe. The lakes of Taupo and Rotorua are also giant calderas. Taupo last erupted in 182AD, coating the North Island in thick layers of ash and pumice.
South Island
The South Island is dominated by the spine of the Southern Alps, a young mountain range that runs most of the length of the island. This range includes New Zealand's highest mountain, Aoraki / Mount Cook (3754m). There are two key cities in the South Island: Christchurch about midway down the east coast and Dunedin near the bottom of the east coast. The town of Queenstown, inland from Dunedin, is also a good base for outdoor adventure.
While lacking the spectacular volcanism of the North Island, the South Island offers the a lot of variety for trampers. The West Coast is relatively wet and rainy, creating rich, diverse, tangled forests, compared with drier beech forests to the east. The southwest corner of the island is dominated by glacier-carved valleys and is particularly wet and inaccessible. Many of the most famous tracks are in this area, including the Routeburn, Milford and Kepler Tracks. The northern tip of the island offers palm beaches, karst landscapes, and ancient grassland peneplains.
There are three road passes over the Southern Alps, each offering good tramping accesss. Furthest south is Haast Pass which leads from near Wanaka through to the West Coast. Tracks in this area are mostly challenging wilderness experiences. Further north is the raw landscape of Arthur's Pass. Arthur's Pass National Park contains a large selection of tracks, mostly maintained to a basic level. Being just a few hours from Christchurch, the park is a good landscape for competent tramping parties to explore. Further north still is Lewis Pass, a beautiful drive through tall beech forests. There are many easier tracks in this area as well as some easier "tops" routes.
Public land
New Zealand has 13 national parks, each preserving outstanding and distinctive landscapes. Tongariro was our first national park and is also a World Heritage Area. In the southwest of the South Island, Fiordland, Mount Aspiring, Westland, and Aoraki/Mount Cook National Parks combine to form another World Heritage Area. Conservation parks are subject to fewer regulations than national parks and provide mixed use opportunities -- biking, horse and 4WD trekking for example. They also provide excellent scope for tramping. Some other public lands are unclassified, but still available for tramping. Access to all government public lands and tracks is free of charge: there are no park fees or permits required.
Department of Conservation
The Department of Conservation (DOC) manages the public land estate as well as an extensive infrastructure of tracks, bridges, huts, and other facilities. DOC visitor centres are found throughout the country in towns and national parks. DOC can provide information and advice on tracks, weather, and current conditions.
Huts
New Zealand has over 1000 back country huts. Most are operated by DOC, although some are owned by clubs -- often these are also available for use. Huts do not have food available for purchase and generally do not have cooking supplies or facilities. Many huts on the more popular tracks will require booking in advance. However, the majority are available on a first-come first served basis.
Find out more about huts: Beginners' guide to... Backcountry Huts.
Camping
Camping is freely available on any backcountry land with a few restrictions, such as right next to huts. Campfires are generally discouraged as they pose a fire risk, strip the forest of rotting wood, and leave messy fire circles. In some areas they may be prohibited.
If no toilet facilities are available, take care to bury waste. Toilet paper takes a long time to biodegrade.
Tracks
The Great Walks are a set of premium tracks, maintained to a high standard, and offering good facilities and transport. These are a great place to start if you are new to the country. Avoid the temptation to save money and walk these in the off season. Off season Great Walks are not for inexperienced trampers, with ice and avalanche risks, bridges removed, and reduced facilities. It's not worth the risk.
Most tracks are not maintained to Great Walks standards and are much more rudimentary. Typical tracks are rough underfoot, require some back country experience, and at least lightweight tramping boots. Track signage is typically green and gold, with small orange triangles marking the way.
Safety
The key tramping hazards are drowning in rivers, hypothermia, and falls. Know when not to cross a river, and pack gear that will keep you warm and dry in the worst weather. Unfortunately, stories of overconfident and inexperienced visitors drowning in rivers or falling to their deaths on wet snowgrass slopes are all too common. Read more: Beginners' guide to...staying safe.
Getting prepared
Larger towns and cities all have outdoors stores that can sell you packs, tents, sleeping bags, cookers, boots, thermal clothing, and other gear. Common brands are: Torpedo 7, Macpac, Bivouac, Kathmandu.
Campstove fuel is readily available at outdoors stores. Standard propane/butane canisters are also available at hardware stores (Bunnings, Mitre 10) and big box stores (The Warehouse). MSR has an information page on stove fuels: http://www.cascadedesigns.com/msr/blog/fuels-around-world-finding-stove-fuel-foreign-country/
Finally...
Visitors to New Zealand often underestimate the ruggedness of the landscape and the variability of the weather with fatal consequences. Easy routes and river crossings can become dangerous or impossible in hours. Take a map and know how to read it. Be prepared to change your plans. Take spare clothing and a raincoat with you even on short walks. Let somebody know where you are going and when to call the police if they don't hear back.