The singer marries a boarding school bred lady who does nothing but read novels. One day he catches her in the cellar "paying the rent." "I'd rather marry the devil than wed with a boarding school lass." "So beware of the Finnegan lasses"
For a definition of "finikin," specifically "precise in trifles, idly busy," see TheFreeDictionary site. The word is like current "finicky."
There is also a parody at Bodleian, Harding B 11(1206), "The Finiken Man," J. Pitts (London), 1819-1844 - BS