Generally no. A straight Lithium 12-volt battery will not be 12 volts. A 12v lithium LiFePO4 battery fully charged to 100% will hold voltage around 13.3-13.4v. Its lead-acid cousin will be approx 12.6-12.7v. A lithium battery at 20% capacity will hold voltage around 13V, its lead-acid cousin will be approx 11.8v at the same capacity. As you can see, we are playing with a very narrow window of voltage with lithium, less than 0.5V over 80% capacity.
A Lithium LiFePO4 charger is a voltage-limiting device that has similarities to the lead-acid system. The differences with Li-ion lie in a higher voltage per cell, tighter voltage tolerances, and the absence of trickle or float charge at full charge. While lead-acid offers some flexibility in terms of voltage cut-off, manufacturers of LiFePO4 cells are very strict on the correct setting because Li-ion cannot accept overcharge. They could otherwise catch on fire and they burn very hotly because they give off oxygen as the burning chemicals break down.
Modern Lithium batteries for use in caravans and cars have expensive electronics built-in that can compensate for these charging differences, but you need to make very sure this is present before using a lead-acid type charger.
Lithium batteries for computers have these controls built into either the charger or the laptop, and must only ever have the correct exact charging voltage.
Please see below for an explanation of the differences between a Lead Acid profiled charger vs a charger that has a Lithium specific profile, and what the implications are:
A Lithium charge profile vs. a Lead Acid profile usually has a slightly higher charge voltage and a "deeper" constant voltage phase at the end of the charge cycle, which is able to get the Lithium batteries to a full 100% State of Charge. A Lead Acid profile will not normally be able to achieve this, however, it will still get close, say maybe 95% +. Research has shown that you can extend the life of Lithium batteries if you prevent charging to 100%, however, the effect only has a pronounced impact if charged to less than 80%. Between 80% to 100% State of Charge (SOC), there is not much observable difference to the battery life.
The charge profiles have different "return to boost" behavior. After the charger completes its full charge cycle (i.e. it reaches the end of the absorption phase and changes over to the float stage), the charger will then start to monitor the battery voltage, to decide when to switch back to the boost/bulk charge mode and restart a full charge cycle again. The issue is that most Lead Acid chargers will wait for the voltage to drop too low before it restarts the charge cycle - that is because a Lead Acid battery has a lower resting voltage than a Lithium battery.
The implications are that the Lithium batteries' State of Charge may already be quite low before the Lead Acid charger decides to switch back into a full bulk charge again. On the contrary, a Lithium profiled charger will start to recharge the Lithium battery much sooner. The Lead-Acid charger will eventually switch back into a full charge cycle again, but your battery may not have changed much yet when you would expect it to be fully charged, and when charging from solar this could mean that energy is wasted that could have been used to top up the batteries because the Lead Acid charge profile of the solar controller may not have recognized that the batteries need charging.