Serious Attempt to get into X4 as a Totally Blind Player

[CBJ]

There is a UI scale option in the in-game settings interface. No need to go fiddling around in the xml.

The option you found there, incidentally, is for the volume of UI sounds. While it may be useful to you, it won't help your OCR software!

[oasis1701]

There is a UI scale option in the in-game settings interface. No need to go fiddling around in the xml.

The option you found there, incidentally, is for the volume of UI sounds. While it may be useful to you, it won't help your OCR software!

Thank you, i thought i had to drag the slider with mouse, forgot that the game has a great keyboard navigation controls.
I was wondering if you guys read my post about the tolk library that i wrote couple of pages ago, even a very limited access that could read the values would help us quite a lot.

[CBJ]

We did read it, and we're investigating. Please bear in mind, however, that even working out whether these things are viable can take time, no matter how "simple" the API providers say they are to integrate.

[kuertee]

i've updated the mod. details at the end of this post.
i didn't include an upscaled font - hoping that the game's native UI scaling is sufficient.
but i did add two blank rows to start the navigation computer menu. hopefully this alleviates the problem of the screen reader not detecting the first line of the menu.

my next steps are:
(1) to get hiring npcs accessible. at the moment, the UI to assign an npc their position requires interaction with a window on the right side of the map screen, then a right-click of a row on the left side, then a left-click on a drop-list on the right, then a click on the confirm button. LOL.
to make this accessible, i'll present the steps over two list (after you choose to hire the npc from the conversation menu, of course): station and ship list to choose one, then a position list to assign to them.
when i get this working, it'll be easy to convert the current ship commands of "autopilot to", "dock at" etc. into ai orders. you won't need to take control of the ship.
also with an ai pilot, i can add an "attack" command.
(2) mission list. when a mission is clicked, it will open the default briefing screen. it's a simple menu that, hopefully, the screen reader can process.

then after that ... sheesh:
(3) buy ships (with one of the default loadouts) from wharves and shipyards.
wharves and shipyards are stations that are already listed in the nav comp.
the default loadouts are serviceable. these are "minimum", "medium" and "full". i can't remember their names off the top of my head.
(4) station building: again, shortcuts to buying a plot (at a location in the sector determined by the mission or chosen randomly), buying a station with a configuration based on the selected production module.
i'm hoping to get the code that that AI factions use to do most of this work.
i'm hoping that all i'll be doing is to add shortcuts to those functions.

anyway, here's the update:
v1.0.5, 18 Aug 2021:
-Tweak: Enter Airlock of X (to return to your ship) now puts you in control of the ship directly. There's a 3 second delay before this teleports you to the controls. The code needs to wait for the AI pilot to leave the seat. If you still don't have control of the ship after, try exiting the airlock and re-entering it.
-Tweak: The Navigation Computer hot key is now usable at any time. In previous version, it was only usable when no menu was open - e.g. while docked. In this version, the hot key will now open the Navigation Computer even when docked.

[oasis1701]

Thanks kuertee, With last night's update we can go around and have fun in the game.
Is there any start that would give me a ship capable of mining?
I always start with young gun; his ship can't mine i think.

[CaplinCaplin]

kuertee,

Your description of the convolutions required to hire crew members amused the hell out of me. Thanks so much for this update, and the to-do list seems like an excellent road map going forward.

I will try my hand at some generic missions from the map for the moment.

Unfortunately, I ran into a situation where my hotkey for the mod fails again. I am not sure if it's a situation with the pipe server, or something in the MD script. I'll see if I Can get a debug log for you. It seems to happen after disembarking at a station, but as with all the best bugs it's not 100% reproducible.

[Falcrack]

My hat's off to you guys, I close my eyes trying to imagine how I would play X4 without being able to see, and it would require an incredible level of persistence to make it happen!

[kuertee]

...Your description of the convolutions required to hire crew members amused the hell out of me. Thanks so much for this update, and the to-do list seems like an excellent road map going forward. ..

admittedly, i have no complaints about it.
its both perfectly usable and uses the generally accepted forms of user interface: left-click to select, right-click to enact, left-click on confirm/cancel/close buttons.
it's just not accessible.

more than a year ago (before i started modding the game), i suggested that players should try to re-frame the game as a pausable first-person-space-captain-and-real-time-strategy-empire-and-economy-building-simulator game (LOL! e.g. the classic Carrier Command on Amiga and Battlezone 2 on PC) because the UI (input and menus) has those elements. left-click to select, right-click to enact, pause to manage your empire. it wasn't well-received by the community, of course, because, at the time, the general public (i.e. new players) expected an Elite-like.

...Unfortunately, I ran into a situation where my hotkey for the mod fails again. I am not sure if it's a situation with the pipe server, or something in the MD script. I'll see if I Can get a debug log for you. It seems to happen after disembarking at a station, but as with all the best bugs it's not 100% reproducible....

there may be two reason for this:
(1) the game is paused when the nav comp is open and my code failed to unpause it when it closed. simply press Esc a couple of times. a clue that this is the case, is if you don't hear the environment sound effects.

(2) my nav comp tried to query an invalid property of an object. (e.g. "destination" is an invalid property of an npc. this example would never happen, of course. its just an example.)
this is a bit more difficult to solve - especially because you won't be able to open the nav comp until the problematic object stops getting listed by the nav comp.
maybe ... throttle up and travel for about 10 seconds, or engage travelmode with Shift-1. then try the nav comp again.

(3) try to open the nav comp from the Interact Menu. simply press T to target something in space. then select the Navigation Computer from the Interact Menu that opens.

[CaplinCaplin]

kuertee,

it looks like the issue is something to do with trying to get null properties on gate components, which then spirals into passing null values to Lua functions which expect strings. I will try your suggestions and be careful about how I open the nav comp, it seems to be more of a problem in the station-based parts of the UI, rather than on ship, at least.

As far as the amount of effort required to play this game, I personally think it's worth the trouble even if the first-person parts of it won't likely be fully playable. There's very little comparable for the blind available, and I appreciate that Egosoft is both willing to look into potential accessibility options themselves, as well as expose modding APIs to make things like the current work possible. Thanks all for the enthusiastic support.

[kuertee]

I've updated the Accessibility Features mod:
Updates:
v1.0.6, 31 Aug 2021:
- New feature: New lists: Owned Stations, Owned Ships, Missions, Sectors In Galaxy.
- Tweak: People is a separate list. Previously, they were listed in the Points Of Interests list.
- New feature: Undock command. Required for autopilot. Previously, the autopilot command was available even when the ship was docked.
- New feature: Retrieve Ship From Interal Storage command. Previously, you can enter a ship that was in an internal storage.
- New feature: Hire / Work Somewhere custom menus. The process of hiring a new employee and transfering a current employee is guided with 2 new custom menus: Location then Position. This process requires that the conversation was started with the Navigation Computer. Otherwise, the usual inaccessible Map Menu is used to complete the hiring and transfer process.
- New feature: During the hiring / transfering process, all of the NPC skills are listed.
- New feature: Repair command on repair panels, satellites and signal leaks. Spacewalking is required to repair objects.
- New feature: Destroy Mines command. When selected, all mines in the mission area are destroyed. Be warned that the autopilot usually stops the ship inside the minefield. I suggest that you boost away (with the Tab key) for a second after the ship stops. Then use the Navigation Computer to destroy one of the mines.
- New feature: Teleport to any of your ships and stations by selecting the Enter Airlock of X command.
- Tweak: Only the Navigation Computer is available from the Interact Menu. All other commands that the previous versions of the mod added were removed because they were sometimes invalid. Also, they were available, and more accurately, from the Navigation Computer.
- Tweak: Autopilots to object within 20km is instant.
- Tweak/bug-fix: The mission target list is now generated from the active mission. Previously, the list was generated from objects that were tagged as a mission target - which produced inaccurate targets.
- Bug-fix: Superhighways out of the sector were getting listed incorrectly.
- Tweak: Better list of valid commands. Previously, commands that could not be actioned were listed. E.g. Autopilot while you were disembarked at a station.
- Bug-fix: Talking to NPCs. Some NPCs simply cannot be visited without actually walking to them. In this version, NPCs that can be teleported to will be teleported to before the conversation starts. NPCs that cannot will use the video com.
- Tweak: Target command is merged with the Turn To Face command.
- Tweak: I rewrote and added sections to this read-me.
- Note: The new version of the Accept mission for later mod is required.

...it looks like the issue is something to do with trying to get null properties on gate components, which then spirals into passing null values to Lua functions which expect strings. I will try your suggestions and be careful about how I open the nav comp, it seems to be more of a problem in the station-based parts of the UI, rather than on ship, at least....

I did fix several bugs in the code that stopped the navigation computer from working. Hopefully, I didn't miss any others.

Just a reminder that the Navigation Computer does Pause the game. And, although I unpaused the game when the menu closes, there may be instances when the game remains paused.
You'll be able to tell because no audio will come from the game.
If this is the case, unpause the game by pressing ESC until the game is unpaused.

[CaplinCaplin]

Edit: Item 3 seems to be working properly, though I had trouble actually deploying a satellite at the requested location. I was directed to what seemed to be the proper place though. I also had occasion to try destroying mines, which worked the second time around, after one game-over as you warned might happen.

Hi there,

I just wanted to say thank you for this update. It's excellent as always, and I was able to complete my first transport mission, which went quite well, and successfully hire crew. I do have a few issues I wondered about, if I might…

1. After I finished the transport mission, I noticed that all the future NPCs I spoke with listed the name of the NPC I'd transported, who presumably no longer exists. I am unsure if their skills were entirely accurate, but the dialog at least worked as I expected.
2. I tried to autopilot to Grand Exchange I from somewhere and ended up taking a giant loop around, which seemed to start and end in Argon prime. I ended up just cancelling autopilot after two or three such loops, and choosing to head somewhere else instead.
3. I tried to take a "Drop," mission to deploy a satellite somewhere, but the mission targeting logic didn't seem to recognize where I needed to go. Perhaps this type needs special handling.

I really appreciate your continued work on this. I know it's a super niche mod, but I'm already deriving a lot of enjoyment from the game, and it's is entirely responsible for that.

[kuertee]

Edit: Item 3 seems to be working properly, though I had trouble actually deploying a satellite at the requested location. I was directed to what seemed to be the proper place though. I also had occasion to try destroying mines, which worked the second time around, after one game-over as you warned might happen.

yeah, some satellite deployment missions are difficult normally - let alone for you guys. e.g. satellite coverage missions require that you blanket a wide area of satellites. and estimating how far apart to put your satellites is and where gaps exist make for a difficult mission to complete. i have a mod called Waypoint Fields for Deployment (https://www.nexusmods.com/x4foundations/mods/585) that adds waypoints for the player to follow and drop satellites at. But that mod is not very accessible. I'll need to add accessibility to that mod.

I suggest that you take satellite missions that do not cover area - but require only a set number of satellite drops. With these satellite drop missions, you can drop the required number of satellites anywhere in the mission area. I'm pretty sure that the autopilot to these mission areas will fly the ship to the middle of it. I'll check later.

[*] After I finished the transport mission, I noticed that all the future NPCs I spoke with listed the name of the NPC I'd transported, who presumably no longer exists. I am unsure if their skills were entirely accurate, but the dialog at least worked as I expected.

Oh, I'll check that. Was the transport mission a taxi mission and not a "deliver engineer/manager/pilot/marine" mission?

[*] I tried to autopilot to Grand Exchange I from somewhere and ended up taking a giant loop around, which seemed to start and end in Argon prime. I ended up just cancelling autopilot after two or three such loops, and choosing to head somewhere else instead.

Unfortunately, that happens in the base game also. My mod sets the "middle" of the sector as the autopilot target. But as in the base game, sometimes the autopilot gets stuck in the main superhighway that circles through the major sectors.

I suggest that once you hear your ship computer announce your sector destination, stop engines with the backspace. I recommend that you remap the stop engines key to the Alt key. Stopping engines will also take you out of the superhighway or disengage travel mode and turn off the autopilot.

I experienced this a lot in the early days of my X4 gaming - when I wanted to check out as much as I can. I hardly experience this now likely because my gameplay is more concentrated in building up my relations with one faction.





Which leads me to my question: Does the screen reader parse the Player Information screen properly?
Also, can you step me through how you use the screen reader? I think I asked this before, but I can't remember if that was on reddit or on this forum.
I.e. list down steps on how turn orn the screen reader on Windows 10, and how you trigger it to read the game's screen.

[oasis1701]

Which leads me to my question: Does the screen reader parse the Player Information screen properly?
Also, can you step me through how you use the screen reader? I think I asked this before, but I can't remember if that was on reddit or on this forum.
I.e. list down steps on how turn orn the screen reader on Windows 10, and how you trigger it to read the game's screen.

Going to be honest here -- as your mod and work begins to become excellent in each update, my desperation to have our screen reader directly interfaced with the game grows as well lol. I can only hope that x4 developers and executives are watching the topic as well, to perhaps find a solution for us. A few of my friends are asking me about the progress that's going on here and i'm suggesting them to hold on a bit longer so we can make sure the game is in a playable state and not frustrating to work with.

the player information window -- and the ship information and basically anything that is a table in the game, or multi windowed is a little bit hard to read for us because of the way the OCR detects the text and then lays it out in a disorganized way.
Just an example of how it could be like this, (not from the game.
Actual text:
ship1, 15000, 500, select, delete
ship2, 20000, 1000, select, delete
ocr result:
ship1
ship2
15000
20000
select
500
1000
delete
select
delete
And as the screen becomes more complex, the OCR result can get more disorganized as well, and a bit tedious to comprehend.
How to download and use the screen reader that we use in the next post
edit: Once i go home i'll paste an OCR result of the player information screen

[oasis1701]

Here's how to download the screen reader we use and use it. i'll try to write a short post.
download: https://www.nvaccess.org/download/
Once you have the exe file, you can either run it as a portable app or install it on your computer. Once the app is launched, it might show you a little window asking you if you want to change your keyboard layout, or have the app launched at logon screen or when windows start. you can ignore all of that and press OK.
When the app is running it's down in the notification area and the screen reader is running in the background. y'll notice that when you move your mouse over windows elements it will read the items below it.
Let's say there are three modes of working with the computer with the screen reader.
1. application mode. You can use your arrow keys and basically the windows commands and you'll hear the reader read the elements for you as they will get focused. the primary keys are the arrow keys, tab+shift+Tab keys, space and enter. e.g. you can press windows+m, use arrow keys to navigate the desktop, enter to activate one of the icons. This is the mode that we hope could be possible to play x4 on.
2. Browse mode, (irrelivant to our case) but this is used on browsers and web apps and frames such as discord. the screen reader creates a cursor and as you use the arrow keys it will scroll through links and headings and other web elements. and there are shortcuts to jump around these elements.
3. advanced mode. we use this mode to treat x4 as a single object in the screen, and then perform an OCR on it and after it gives us the OCR results, we use the screen reader functions to put the mouse on where it finds the texts extracted from the screen.
When the game is open, you'll see that when using the keyboard the reader is not able to read the items. find a simple screen with text and perform these commands.
(the insert key is the screen reader modifier by the way. for example, if you hold the insert key+f12, you'll hear the time.
(I also do appoligize if this is hard! Unfortunately it's the way we interact with x4 screens and we have to do it every time there's a new text on the screen in the game)
So once the game is open on a simple screen:
1. make sure numpad is off so it does not input numbers as you press them. then, insert+numpad 1 -- switches the advanced mode to object review. you can make sure it is in object view by pressing insert+numpad 7, (switches to screen review) and then back to object view by pressing insert+numpad 1, once this mode is selected, you don't need to do this again in your next OCR attempts.
2. press insert+r -- screen reader begins scanning the screen and performs OCR. you'll hear, "recognizing... result document"
3. Now there's a hidden interface open and your keyboard focus is inside it. use numpad 7 and 9 to read the previous/next line, 4 and 6 for words, and 1 and 3 for letter navigation.
4. Once you find the text you're interested to interact with, make sure your focus on it by just hearing the text as you focus on it.
5. press insert+numpad slash -- this will drop your mouse pointer on top of the UI element the screen reader has found the text from.
6. press numpad slash to emulate a mouse click or juse use your left mouse button to click.
7. use the escape key to exit the OCR results window, otherwise your left and arrow keys, enter button, the numpads won't work in the game, as they're being captured to browse the ocr result.

I just hope that you do have a keyboard with a numpad -- otherwise Caplin might have to help with the alternate keys for keyboards that don't have one.
You can exit the screen reader by openning its menu from the notification area or pressing insert+n to open the menu and choosing exit. let me know if you had questions.
edit: you can use the command control+insert+up and down arrow keys to adjust the TTS speed rate
edit2: when NVDA is running and your numpad is turned off (as in, it is on page up/down/home/end mode) your entire numpad belongs to NVDA! for advanced navigation. so it will steal and capture them, so you can't use it to press home or end, for example
edit3: some texts in the game are skipped by the OCR, i'm not sure why. for example when the mission window is open, or when trading, some small texts gets skipped. maybe because there's a shadow on them, or something like that.

[kuertee]

Thank you.
I have experience with JAWS and Windows Eyes. But that was many years ago. I was a developer with a computer-based training company. And all our compliance training courses needed to be accessible.
I'll try NV Access tonight.

EDIT:
A quick question: do you think NV Access can read an external file as you play X4?

Spoiler

Show

If so, *I think* I can send text to a file.
E.g. when the Navigation Computer opens, I can send the non-interactable text like the NPC skills stat to the text file.
Then if you use the keyboard to move through the interactable elements of the Navigation Computer (e.g. buttons), I can send that to the text file.
Do you think NV Access can detect changes in that text file and read it without you having to leave X4?

EDIT2: Nevermind my previous edit.
I think NV Access's Insert-7 and Insert-9 are sufficient in navigating through the screen.
Well, I think so, anyway.
We just need to make sure that the screens ARE navigable by NV Access.

[CaplinCaplin]

Yeah, it's probably best to just make sure the screens are as good as they can be with OCR, which judging from my experience so far is pretty good. External files would work but also require add-ons which would need to be written in Python, if we wanted them read automatically. Back when I was trying to mod x3 I did output some things to files which I could read manually, and that did work reasonably well, but x4seems better with OCR anyway, not to mention more modifiable.
Re: your question about the mission and NPC names, it was definitely a taxi (fast flight) mission. I noticed the issue several times now, it seems to repro reliably. Just take and fulfill a taxi mission and then talk to other NPCS on station or whatever.

[oasis1701]

May I ask what you guys mean by external files?
Like an alternative interface? Wouldn't that require massive amounts of work?
Or just a notepad file that we could read information on.
edit: oh i just re read the posts and realize you did mean something like a notepad. It would have helped with better readability but would add an extra step to what we do I guess,

[kuertee]

how do you guys read the load game screen?
NV Access only reads the time of the saved game and not the name of the saved game.
i suppose you guys wouldn't have known that the load game screen contains more info than the file's timestamp
(unless i'm mistaken, of course.)

[oasis1701]

how do you guys read the load game screen?
NV Access only reads the time of the saved game and not the name of the saved game.
i suppose you guys wouldn't have known that the load game screen contains more info than the file's timestamp
(unless i'm mistaken, of course.)

Yeah you're correct. some texts are just skipped with the OCR in the game. that's one of them. What i'll do is try to have only one or two save files so it's easier to load them
edit: BTW, when you have the OCR result open, you can do control+a, control+c, and paste it in a notepad or anywhere you'd like to see the result visually

[CaplinCaplin]

how do you guys read the load game screen?
NV Access only reads the time of the saved game and not the name of the saved game.
i suppose you guys wouldn't have known that the load game screen contains more info than the file's timestamp
(unless i'm mistaken, of course.)

I just make liberal use of "continue," and "quicksave," and try not to have too many save games, as Oasis1701 suggests. It's definitely annoying, but at least we can reliably click on what we need to most of the time.