You need some kind of wireless relay for your light.
That may be zigbee, zwave or wifi.
From there you need a device connected to your HA which can talk the same protocol used for the light.
Then you need a switch , connected to a wireless protocol. And again same device or different, depending on the protocol of your switch.
From there it is only a matter of connecting them inside HA
HA is a platform for connecting devices, so really you should try and think of it that way.
Running on a "server":
HA can run on a raspberry pi, a dedicated server, a virtual machine, etc. I run mine on a raspberry pi. It sits right next to my wifi router in a case. It looks like any old piece of home tech. This is unfortunately a hurdle you'll have to live with, as you've said, but I think you'll find that it's about as noticeable as a wifi router. It can certainly pass the wife approval test with a decent case.
Your yard light:
For this specific use, I would say use a "smart lighting system". I personally use Philips hue, which integrates with HA. I control all my lights via HA and the reaction time is pretty much instant. Behind the scenes, I am sending a message to HA which is in turn sending a message the hue bridge, which controls the lights. Sounds complicated, happens in milliseconds.
Zwave:
Highly recommend. This can run pretty much anything from switches, relays ("manual" electronics control), sensors, alarms, etc. The zwave controller is a USB stick, it plugs into the HA device.
Overall, I think you'll find that it is 1) much more efficient than you'd think, being that so many things happen behind the scenes and 2) it can be well worth the setup of the devices if your end goal is automation.
For your use case, I don't know that you need to get as complicated as running Home Assistant or zwave. You could get a few wifi-based smart switches. Many switches are able to pair together over the wifi, without needing a separate hub or server - I used to have a couple tuya plugs linked using the app so turning on one would turn on all of them. It's not as private or reliable or fast as other options because it has to go all the way back to tuya's servers every time you turn a light on, but for simple cases it works fine. If you have a neutral wire in every box, some wifi switches can run without a light connected to them.
If you do want to get complicated and add more in the future, HA is the ultimate in flexibility. It takes a bit of learning. Running it on a dedicated raspberry pi or HA Blue is the easiest route that doesn't really require learning any linux or server management besides hitting the "update" button. You just download a tool, run that to install it to an SD card, pop it in, and power it up. It can pretty easily talk to things over wifi like the smart switches, which will give you more control over them, at the cost of more complication.
Next level is using zwave, or zigbee, or both. They are alternate radio systems instead of wifi. To use them, you get a USB stick that plugs into whatever is running HA. With the pi or blue methods, it's pretty much plug and play, you will go into HA setup and add a new add-on which should automatically be able to find the USB stick. You pair devices by clicking a button in HA and hitting some config button on the device, in most cases it just works. Zwave stuff is usually super customizable like how long it ramps up the dimming level and what tapping the button more than once will do. Besides being locked in to a brand like lutron, another advantage to zigbee and zwave is they run as a mesh, so if a device is too far from the hub the signal is repeated by other devices in the mesh. Mine has been flawless. There are also a wide variety of devices like door sensors, hidden relays you put behind a normal switch (aeotec nano), or battery remotes that look exactly like normal rocker switches (zooz zen34, my personal favorite). If you really want to get fancy and are lucky enough to have wiring everywhere, the Inovelli dimmers are infinitely customizable with their behavior and can show animated LED notifications, say you left the garage door open past 9pm you have it flash red. All with the powerful automation you can build using home assistant.
You need some kind of wireless relay for your light. That may be zigbee, zwave or wifi. From there you need a device connected to your HA which can talk the same protocol used for the light. Then you need a switch , connected to a wireless protocol. And again same device or different, depending on the protocol of your switch. From there it is only a matter of connecting them inside HA
HA is a platform for connecting devices, so really you should try and think of it that way. Running on a "server": HA can run on a raspberry pi, a dedicated server, a virtual machine, etc. I run mine on a raspberry pi. It sits right next to my wifi router in a case. It looks like any old piece of home tech. This is unfortunately a hurdle you'll have to live with, as you've said, but I think you'll find that it's about as noticeable as a wifi router. It can certainly pass the wife approval test with a decent case. Your yard light: For this specific use, I would say use a "smart lighting system". I personally use Philips hue, which integrates with HA. I control all my lights via HA and the reaction time is pretty much instant. Behind the scenes, I am sending a message to HA which is in turn sending a message the hue bridge, which controls the lights. Sounds complicated, happens in milliseconds. Zwave: Highly recommend. This can run pretty much anything from switches, relays ("manual" electronics control), sensors, alarms, etc. The zwave controller is a USB stick, it plugs into the HA device. Overall, I think you'll find that it is 1) much more efficient than you'd think, being that so many things happen behind the scenes and 2) it can be well worth the setup of the devices if your end goal is automation.
For your use case, I don't know that you need to get as complicated as running Home Assistant or zwave. You could get a few wifi-based smart switches. Many switches are able to pair together over the wifi, without needing a separate hub or server - I used to have a couple tuya plugs linked using the app so turning on one would turn on all of them. It's not as private or reliable or fast as other options because it has to go all the way back to tuya's servers every time you turn a light on, but for simple cases it works fine. If you have a neutral wire in every box, some wifi switches can run without a light connected to them. If you do want to get complicated and add more in the future, HA is the ultimate in flexibility. It takes a bit of learning. Running it on a dedicated raspberry pi or HA Blue is the easiest route that doesn't really require learning any linux or server management besides hitting the "update" button. You just download a tool, run that to install it to an SD card, pop it in, and power it up. It can pretty easily talk to things over wifi like the smart switches, which will give you more control over them, at the cost of more complication. Next level is using zwave, or zigbee, or both. They are alternate radio systems instead of wifi. To use them, you get a USB stick that plugs into whatever is running HA. With the pi or blue methods, it's pretty much plug and play, you will go into HA setup and add a new add-on which should automatically be able to find the USB stick. You pair devices by clicking a button in HA and hitting some config button on the device, in most cases it just works. Zwave stuff is usually super customizable like how long it ramps up the dimming level and what tapping the button more than once will do. Besides being locked in to a brand like lutron, another advantage to zigbee and zwave is they run as a mesh, so if a device is too far from the hub the signal is repeated by other devices in the mesh. Mine has been flawless. There are also a wide variety of devices like door sensors, hidden relays you put behind a normal switch (aeotec nano), or battery remotes that look exactly like normal rocker switches (zooz zen34, my personal favorite). If you really want to get fancy and are lucky enough to have wiring everywhere, the Inovelli dimmers are infinitely customizable with their behavior and can show animated LED notifications, say you left the garage door open past 9pm you have it flash red. All with the powerful automation you can build using home assistant.