game design - How to develop RPG Damage Formulas?

I'm developing a classical 2d RPG (in a similar vein to final fantasy) and I was wondering if anyone had some advice on how to do damage formulas/links to resources/examples? I'll explain my current setup. Hopefully I'm not overdoing it with this question, and I apologize if my questions is too large/broad

My Characters stats are composed of the following:

enum Stat
{
    HP = 0,
    MP = 1,

    SP = 2,
    Strength = 3,
    Vitality = 4,
    Magic = 5,
    Spirit = 6,
    Skill = 7,
    Speed = 8, //Speed/Agility are the same thing
    Agility = 8,
    Evasion = 9,
    MgEvasion = 10,

    Accuracy = 11,
    Luck = 12,
};

Vitality is basically defense to physical attacks and spirit is defense to magic attacks.

All stats have fixed maximums (9999 for HP, 999 for MP/SP and 255 for the rest). With abilities, the maximums can be increased (99999 for HP, 9999 for HP/SP, 999 for the rest) with typical values (at level 100) before/after abilities+equipment+etc will be 8000/20,000 for HP, 800/2000 for SP/MP, 180/350 for other stats

Late game Enemy HP will typically be in the lower millions (with a super boss having the maximum of ~12 million).

I was wondering how do people actually develop proper damage formulas that scale correctly? For instance, based on this data, using the damage formulas for Final Fantasy X as a base looked very promising. A full reference here http://www.gamefaqs.com/ps2/197344-final-fantasy-x/faqs/31381 but as a quick example: Str = 127, 'Attack' command used, enemy Def = 34.

1. Physical Damage Calculation:
Step 1 ------------------------------------- [{(Stat^3 ÷ 32) + 32} x DmCon ÷16]

Step 2 ---------------------------------------- [{(127^3 ÷ 32) + 32} x 16 ÷ 16]
Step 3 -------------------------------------- [{(2048383 ÷ 32) + 32} x 16 ÷ 16]
Step 4 --------------------------------------------------- [{(64011) + 32} x 1]
Step 5 -------------------------------------------------------- [{(64043 x 1)}]
Step 6 ---------------------------------------------------- Base Damage = 64043
Step 7 ----------------------------------------- [{(Def - 280.4)^2} ÷ 110] + 16
Step 8 ------------------------------------------ [{(34 - 280.4)^2} ÷ 110] + 16
Step 9 ------------------------------------------------- [(-246)^2) ÷ 110] + 16
Step 10 ---------------------------------------------------- [60516 ÷ 110] + 16
Step 11 ------------------------------------------------------------ [550] + 16

Step 12 ---------------------------------------------------------- DefNum = 566
Step 13 ---------------------------------------------- [BaseDmg * DefNum ÷ 730]
Step 14 --------------------------------------------------- [64043 * 566 ÷ 730]
Step 15 ------------------------------------------------------ [36248338 ÷ 730]
Step 16 ------------------------------------------------- Base Damage 2 = 49655
Step 17 ------------ Base Damage 2 * {730 - (Def * 51 - Def^2 ÷ 11) ÷ 10} ÷ 730
Step 18 ---------------------- 49655 * {730 - (34 * 51 - 34^2 ÷ 11) ÷ 10} ÷ 730
Step 19 ------------------------- 49655 * {730 - (1734 - 1156 ÷ 11) ÷ 10} ÷ 730
Step 20 ------------------------------- 49655 * {730 - (1734 - 105) ÷ 10} ÷ 730
Step 21 ------------------------------------- 49655 * {730 - (1629) ÷ 10} ÷ 730

Step 22 --------------------------------------------- 49655 * {730 - 162} ÷ 730
Step 23 ----------------------------------------------------- 49655 * 568 ÷ 730
Step 24 -------------------------------------------------- Final Damage = 38635

I simply modified the dividers to include the attack rating of weapons and the armor rating of armor.

Magic Damage is calculated as follows: Mag = 255, Ultima is used, enemy MDef = 1

Step 1 ----------------------------------- [DmCon * ([Stat^2 ÷ 6] + DmCon) ÷ 4]
Step 2 ------------------------------------------ [70 * ([255^2 ÷ 6] + 70) ÷ 4]
Step 3 ------------------------------------------ [70 * ([65025 ÷ 6] + 70) ÷ 4]
Step 4 ------------------------------------------------ [70 * (10837 + 70) ÷ 4]

Step 5 ----------------------------------------------------- [70 * (10907) ÷ 4]
Step 6 ------------------------------------ Base Damage = 190872 [cut to 99999]
Step 7 ---------------------------------------- [{(MDef - 280.4)^2} ÷ 110] + 16
Step 8 ------------------------------------------- [{(1 - 280.4)^2} ÷ 110] + 16
Step 9 ---------------------------------------------- [{(-279.4)^2} ÷ 110] + 16
Step 10 -------------------------------------------------- [(78064) ÷ 110] + 16
Step 11 ------------------------------------------------------------ [709] + 16
Step 12 --------------------------------------------------------- MDefNum = 725
Step 13 --------------------------------------------- [BaseDmg * MDefNum ÷ 730]
Step 14 --------------------------------------------------- [99999 * 725 ÷ 730]

Step 15 ------------------------------------------------- Base Damage 2 = 99314
Step 16 ---------- Base Damage 2 * {730 - (MDef * 51 - MDef^2 ÷ 11) ÷ 10} ÷ 730
Step 17 ------------------------ 99314 * {730 - (1 * 51 - 1^2 ÷ 11) ÷ 10} ÷ 730
Step 18 ------------------------------ 99314 * {730 - (51 - 1 ÷ 11) ÷ 10} ÷ 730
Step 19 --------------------------------------- 99314 * {730 - (49) ÷ 10} ÷ 730
Step 20 ----------------------------------------------------- 99314 * 725 ÷ 730
Step 21 -------------------------------------------------- Final Damage = 98633

The problem is that the formulas completely fall apart once stats start going above 255. In particular Defense values over 300 or so start generating really strange behavior. High Strength + Defense stats lead to massive negative values for instance. While I might be able to modify the formulas to work correctly for my use case, it'd probably be easier just to use a completely new formula. How do people actually develop damage formulas? I was considering opening excel and trying to build the formula that way (mapping Attack Stats vs. Defense Stats for instance) but I was wondering if there's an easier way? While I can't convey the full game mechanics of my game here, might someone be able to suggest a good starting place for building a damage formula?

Thanks