As there is no Delete option available in Services MMC, it was becoming a nightmare to re-install it. Some how I found SC.EXE which is a command line program to communicate with Service Control Manager and Services.
There are many available options but what interested me is you can Create, Start, Stop, Delete a service with a Command Line.
Creating a Service:sc.exe create PayCalcService binPath= "C:\Program Files\PaymentCalculation\paycalc.exe" DisplayName= "PaymentCalculationService"
Starting a Service:
sc.exe start PaymentCalculationService
Stopping a Service:
sc.exe stop PaymentCalculationService
Deleting a Service:
sc.exe delete PaymentCalculationService
Hope this helps for similar problems...