There is a time-honored minhag going at least back to Kalev (if not the medrash about Rochel Imeinu's kever) to daven AT kivrei tzadikim.
How are you reading the gemara? It (to me) seems pretty clear Kalev is asking the Avos for help.
אָמַר רָבָא מְלַמֵּד שֶׁפֵּירַשׁ כָּלֵב מֵעֲצַת מְרַגְּלִים וְהָלַךְ וְנִשְׁתַּטַּח עַל קִבְרֵי אָבוֹת אָמַר לָהֶן אֲבוֹתַי בִּקְּשׁוּ עָלַי רַחֲמִים שֶׁאֶנָּצֵל מֵעֲצַת מְרַגְּלִים
Rava says: This teaches that Caleb separated himself from the counsel of the other spies and went and prostrated himself on the graves of the forefathers in Hebron. He said to them: My forefathers, pray for mercy for me so that I will be saved from the counsel of the spies.
There is a machlokes if asking the tzadik for help is permitted, but let us say (for argument's sake) that you go with those who permit it.
Yes, as you, I, and others mentioned, it's an old machlokes, but see the tshuva I referenced above that not only is it permitted, but he writes "אדרבא, מצוה איכא"
שו״ת מנחת אלעזר חלק א׳ סימן ס״ח
https://hebrewbooks.org/pdfpager.aspx?req=14912&st=&pgnum=172The problem is "even when Hashem Himself doesn't give a yeshua, Rashbi has the power." If you don't think that is kefira, I can't help you.
I think there's a misunderstanding here. I don't think anyone is forgetting the True Source. Sometimes to get a "favorable" outcome from the True Source different methods may need to be employed, such as tshuva/tzedaka/tfila/tfilas tzaddikim etc.
ETA: Gemara with Kalev is
סוטה ל״ד ב