It's impossible to not wonder at the writing ability of Ondaatje, so why three stars?

The idea of the book is interesting, the effect of a mother's secret life on her children. Yet it is largely the story of women told by a man. Not Ondaatje but the narrator of the story, the son. There's something off that I haven't been able to put my finger on yet.