Each public utility shall have an office within the Virgin Islands in which it shall keep all such books, accounts, papers, and records as shall be required by the Commission to be kept within the Virgin Islands. No books, accounts, papers, or records required by the commission to be kept within the Virgin Islands shall be at any time removed therefrom, except on such condition as may be prescribed by the Commission: provided, that public utilities operating in the Virgin Islands and elsewhere who have their general or executive offices outside of the Islands, may continue to keep their books, accounts, records, and so forth, at their executive or general offices, such public utilities being required, however, to produce before the Commission such books, accounts, records, and papers from time to time as the Commission may order.