Temples are not so rare in Japan. Don't you think so? There are really a lot of temples in Japan. "Japan has 77,206 temples" said the 2016 examination by Agency for Cultural Affairs.
For your reference, there are about 24,000 post offices in 2018 Japan. I guess that it is difficult to regard temples as something rare, therefore. In other words, temples are pretty hard to show the uniqueness each other.
Koen-ji is one of Tokyo temples, which can easily raise their individuality.
It is said that the Koen-ji was very fated to Tokugawa Iemitsu, the 3rd shōgun of the Tokugawa dynasty. However, I don't mean that he built the temple. The temple was originally built in 1555, Iemitsu was born in 1604.
Iemitsu loved hawking so much like his grandfather Ieyasu. Hawking (falconry) is an outdoor hunting game which many old nobles were crazy about.
One day, Iemitsu went hawking, dropped in at the Koen-ji. There were no magazines, photos, internet and televisions in 1600's. Naturally, the chief priest didn't know who Iemitsu was but treated him so well. Iemitsu got fond of the temple. Finally, it is said that Iemitsu planted a thank-you tea tree in the garden by himself.
They say the tree is still there.
In 1945, Tokyo was deep attacked by the B-29 of the United States of America. The temple was also bombed, burnt to cinders. It is said that the present main hall was rebuilt in 1953. It is consoling to think that the tree was kept safe.