The Bank has achieved much in recent years. Inflation is coming down and is below 5%. Interest rates are lower too. Foreign exchange reserves have never been higher, and GDP growth has averaged between 7% and 8% over the last three years.
And yet foreign direct investment (FDI) lags. Last year the government had hoped for some $2bn (£1.2bn) in FDI. It got $1.3bn.
That is why Mr Cabraal is on a mission to market the country to international investors.
On a recent trip to London, accompanied by a delegation of officials and business representatives, he was prepared for questions on human rights to come up.
They nearly always do, but he played it down: "There are a few who have asked the question - but not all. And every time they have asked the question we have explained to them in detail what is taking place and they have been satisfied with our answers."
Sri Lanka's civil war between the northern Tamil Tigers and the government dragged on for 26 vicious years, ending five years ago this month.
In March the United Nations Human Rights Council passed a resolution calling for an independent investigation into allegations of war crimes by all sides during the final months.
Sri Lanka has refused to allow any such investigation.
Meanwhile, Amnesty International insists that the government "continues its crackdown on dissent, which appears to be aimed at intimidating and punishing those who attempt to communicate concerns about human rights violations in Sri Lanka to the United Nations".
I put the question again to Mr Cabraal. Is the issue of human rights a block on investment? Not at all, he replied. He has sold £6bn in Sri Lankan bonds to investors around the world, 70% of them from the US, the EU and the UK. The idea that they have been put off buying because of human rights issues is "entirely untrue".
Ajith Nivard Cabraal, Sri Lanka's Central Bank Governor.
Human rights issues are no block on investment, says Ajith Nivard Cabraal
I suggested that investors who might be happy to put a few Sri Lankan bonds in their portfolio could still balk at doing business on the ground. No, he said. Bond investors were quite aware that their funds would be spent in real projects, in roads and ports - and they had no objections.
Hardy Jamaldeen is the director of Steradian Capital, which advises foreign investors on property investment in Sri Lanka.
The human rights question, for him, is very real: "It's one of the hazards we are fighting on a regular basis.
"I've been here over the past week meeting with a group of core investors and a few of them say, 'I'm very sorry Hardy but we will not be coming in because of these issues.'"
He says the war "did not end particularly well" and that reconciliation needs to happen. But he believes foreign investment can be a power for good - many of his investment opportunities are in the north, in the Tamil heartland.
"I'm saying - go there. Get involved," he says.
"Do it in a proper manner. We are looking at a lot of corporate social responsibility projects... to help schools and communities in [the north]. This is what they need. If you isolate them... actually the problem doesn't get better."
Ring of Pearls?
One group of investors appear to have never raised concerns over human rights - the Chinese.
Their presence is everywhere, financing the new $500m Colombo Port Terminal, Hambantota Port, Sri Lanka's first four-lane expressway, and the National Theatre, to name but a few.
Mr Cabraal believes this is central to the country's economic development.
But others point out how Sri Lanka has passively become part of China's "Ring of Pearls" - a line of military and commercial facilities from the South China Sea through the Straits of Malacca, taking in Pakistan, Bangladesh, the Maldives and Somalia.
Open it may be, but getting through it is hardly easy.
The World Bank has the country ranked 85th in its "Ease of Doing Business" tables. Mr Cabraal brushes this away saying the country was ranked 105th just three years ago. "We have been working away at this very diligently."
Mr Jamaldeen agrees. His business specialises in guiding clients through the red tape. He says the bureaucracy has lessened significantly in the last three years.
But there are still punitive taxes of over 100% for foreigners buying property in the country.
Protectionist? Mr Cabraal thinks not - he doesn't want property prices driven up by foreign speculators taking them out of reach of local people. This kind of protection he thinks is "an excellent situation".
So investors considering putting their money in Sri Lanka need to weigh up human rights, bureaucracy and profits.
And the profits, says Mr Jamaldeen, are real: "I wouldn't look at an opportunity unless you are looking at 30% in terms of IRR (internal rate of return). You can find that in a comfortable environment. There are other trading environments where you are making multiples. There are real opportunities in property."
And it seems the greatest aid to converting investors is the country itself.
"Once we get these investors out to Sri Lanka 94% or 95% are converting," says Mr Jamaldeen. "But actually getting them out is the hindrance - because of the overhang of the UN position."