The highly unusual move, which comes amid a Beijing crackdown on the domestic entertainment industry, was announced in a festival statement citing "technical difficulties encountered during post-production".
Zhang's Yi miao zhong (One Second) had been scheduled to screen on Friday. It is billed as an intimate portrait of a prisoner who escaped from a labour camp during the Cultural Revolution and is on a trek to see a key newsreel in a rural village cinema.
The film's press material noted that Zhang himself "was sent to the country as a young man during the Cultural Revolution, an experience he has explored in many of his films".
"In Yi Miao Zhong, the filmmaker pays homage to his medium and celebrates the cinema as a communal experience which goes beyond the film itself," it said.
The Berlinale festival, one of Europe's top cinema showcases, said it would show an earlier Zhang film instead, 2002's Hero, and that the competition for the Golden Bear top prize would now only feature 16 titles.
A jury led by French Oscar winner Juliette Binoche is to pick the winner on Saturday. Two other Chinese directors, Wang Quanan and Wang Xiaoshuai, are still in the running.
The cancellation follows another Chinese film, Better Days, directed by Hong Kong's Derek Tsang, being withdrawn last week, also for "technical reasons".
It was described as a story about a disaffected youth and was to premiere in the Generation sidebar section.
Zhang, the maker of such classics as Raise the Red Lantern and House of Flying Daggers, is one of China's best-known directors abroad and a frequent guest at prominent European film festivals.
He won the Golden Bear in 1987 with the drama Hong gaoliang (Red Sorghum) starring Gong Li.
Social media reaction to the pulling of One Second
In 2009, he presented A Woman, A Gun, A Noodle Shop, a remake of the Coen Brothers' Blood Simple, at the event. Two years later he screened The Flowers of War featuring Christian Bale.
Insiders in the Chinese film and television industry have recently noted sharper government scrutiny.
The entertainment sector had blossomed in recent years, with official encouragement by a government keen to replace foreign content with home-grown fare and develop the industry as a global "soft power" asset.
But a nationwide push for more material approved by the Communist party across media, music and entertainment has had a chilling effect on some artists.