VNC Keyboard Events

[dustobub]

I've spent several hours on google trying to find a solution to this issue but have found very little info. I'm actually quite surprised as I would have thought eventghost users would have attempted VNC solutions. I'm trying to get HippoRemote for the iPhone to communicate with EG. Hippo communicates via VNC so the underlying issue is a general VNC issue. I assume VNC is using SendInput() or PostMessage() to send keyboard events, but there doesn't seem to be any way to get EG to trigger off of these events as the standard keyboard plugin can't see these events. RDP triggers keyboard events because it uses a virtual keyboard driver rather than the direct functions. I feel this should be relatively simple to implement in a new plugin, but I don't have the expertise to do so. Does anyone know if there is a potential solution with the current plugins? I've tried multiple VNC servers (ultravnc, realvnc) but they all seem to behave the same. Any ideas would be appreciated. EventGhost and HippoRemote would really be a killer combo.

Thanks,
Dustin

[stottle]

I'm confused about how you would expect this to work. VNC, if I understand correctly, gets an image of the screen which it can display remotely. Based on how you interact, it will bring other applications to the foreground and interact with them. It doesn't look like HippoRemote shows a window, so I assume (valid?) that it will only interact with the foreground window.

If so, that doesn't play well with EG. EG is intended to run in the background. Ideally you would have something that sent messages to EG, and let EG decide who to forward to. If EG was in the foreground (so that it received events), how would it know what to do with the events? And wouldn't it mess up playback if EG was in the foreground?

I'm not trying to be rude. HippoRemote looks like a nice app, and I'm a proponent of EG playing well with others. I just don't see how these two techniques play well together. Am I wrong?

Brett

[dustobub]

Thanks for the reply. Definitley not rude, all issues you raised were valid and pointed. Ideally I think I would like a plugin that triggered an event on any VNC sendinput() regardless of the window that VNC was interacting with. Hippo does only interact with the foreground window, but I feel like it should be possible to intercept these events. Is EG not currently designed to provide this kind of process interaction?

Thanks,
Dustin

[dustobub]

If EG was in the foreground (so that it received events), how would it know what to do with the events? And wouldn't it mess up playback if EG was in the foreground?

If VNC only communicates with the foreground window, and Eventghost was able to listen to all SendInput() commands sent to the foreground window, then I think it would work correctly regardless of EG being in the foreground or background. EG wouldn't be trapping the events, just intercepting them, and passing them along to the foreground window.

[stottle]

I'm by no means an expert in VNC, so take this with a grain of salt (and please correct if I'm wrong). My understanding is that its VNC client (HippoRemote) -> VNC Server (HippoServer?) -> Foreground Application. If EG isn't the foreground application, EG isn't in the loop. Unless the VNC Server was modified to pass the info to EG, I think you are out of luck.

At the same time, I saw on the HippoRemote website that the server is free/opensource. Seems like it would be possible to make a plugin for EG that became/replaced the VNC server and then let EG handle the events. That plugin just doesn't exist at the moment.

Brett

[dustobub]

My understanding is that its VNC client (HippoRemote) -> VNC Server (HippoServer?) -> Foreground Application. If EG isn't the foreground application, EG isn't in the loop. Unless the VNC Server was modified to pass the info to EG, I think you are out of luck.

That first section is definitely correct. As for the second, modifying the server side (HippoVNC) code is one possible solution, but I think the more elegant and useful way is to create a more general EventGhost plugin that would listen for these SendInput() events targeting the foreground window. This way, any VNC server/client could trigger events within EG, not just Hippo. I understand this plugin doesn't currently exist, but I am hoping to get some confirmation that this type of plugin is feasible inside of EG.

Thanks,
Dustin

[stottle]

So you want to be able to read SendInput() messages, if they come from VNC applications, and pass them to EG while also passing them to the foreground window?

If so, I suggest you look outside the EG forums. That isn't an EG specific question, that's a Windows OS question and a pretty obscure one. If it is possible to monitor all SendInput() events and tell which app they come from, then I would assume that it would be possible to create a plugin for EG that did that as well. I think you will have more luck finding the answer on an MSDN forum or some such.

Brett

[dustobub]

Yeah, that's pretty much it. I've been in communication with the HippoRemote dev and he has been very supportive. He was interested in getting Eventghost and Hipporemote to play nice together, but he mentioned that AutoHotKey and Launchy both have no issues reading keystrokes sent by VNC servers, and he felt that the existing keyboard plugin could be adapted. Does anyone know who maintains/wrote the keyboard plugin?

Thanks,
Dustin

[dustobub]

I managed to create a workaround using AutoHotKey, but it is most definitely a workaround. I would still love this to be handled natively in EventGhost, but this will work for now. I am having AutoHotKey pickup the VNC keystrokes, and then passing them via comandline to EG like this:

#z::Run C:\Program Files\EventGhost\EventGhost.exe -e test

This will require another whole set of mappings, but it will work.

Thanks,
Dustin

[stottle]

[I was writing this in response to your questions two posts back. That's quite a hack (cringe ) you've come up with.]

You apparently don't like my answer and you seem to be waiting for someone else to chime in with an answer you like more. The author of the original plugin (it gives the maintainer in the plugin .py file, which is plain text by the way) is Bitmonster.

Personally, I think the direction you want to go is a really bad idea. The current keyboard plugin will allow you to respond to keyboard input meant for other apps, but it then doesn't send the input to the other app. Because of that, that plugin is not used often. You may think that modifying the code to give EG the keystroke while still sending it to another app is a better idea - and that probably is possible - but then you can't tell what result you will get. For instance, many players use the spacebar to toggle play/pause. If the player is in the foreground, it would get the command twice, once from the VNC Server and once from EG. Probably not what you want. I believe you want one program to have control, not two. So it should be the VNC server, or EG, but not both.

The premise for EG is to control your computer/environment. That requires knowing what will happen when you take an action. But if multiple applications can respond to events, you can't know what the results will be. You've got an uphill battle if you want to make changes that can cause erratic behavior.

I think that creating a plugin that runs the VNC server and forwards the commands to EG and only EG is a possible answer for what you are looking to do.

Make sense?

Brett

[dustobub]

Sorry, it's not that I didn't like your answer, I was responding from my phone, and I didn't have time to fully respond to all of your points. The other part is that I don't have the knowledge to write a plugin or modify the VNC code, and it would take months for me to feel comfortable doing so. I am looking for a solution that can be put in place sooner rather than later, even if it is a bit hackish.

Also, I was confused about the design of the keyboard plugin. After your explanation and looking into the keyboard description/code, I understand its design. Thank you. I didn't realize that it fullytrapped the keys when triggered. I mispoke earlier, there is no need to pass the event to both the foreground window and EG. Having EG trap the key is fine for what I am looking for, and I understand that sending duplicate commands could cause undesirable behavior if it wasn't handled properly in EG's configuration. Theoretically, it could work if configured correctly, but it is unnecessary.

If the keyboard plugin trapped the SendInput() commands and hardware keypresses, like AHK/Launchy, and then triggered an event, things would work perfectly.

I like your idea about created a VNC build that could configurably send keypresses or EventGhost events, but I think having EventGhost's Keyboard plugin listen to SendInput() like AHK would be a simpler design. Don't you think so?

Thanks for all of your help and sorry for the confusion,
Dustin

[stottle]

That makes sense. Thanks.

If the keyboard plugin trapped the SendInput() commands and hardware keypresses, like AHK/Launchy, and then triggered an event, things would work perfectly.

What you want is to trap keypresses, but only if they come from HippoRemote. Is there a way to do this? I thought that is you trapped keypresses, you trapped them all. And you don't want to trap them all, right? Any idea what AHK or Launchy do? You distinguish between SendInput and hardware keypresses, but I thought they looked the same at the point you can trap. Am I missing something?

I like your idea about created a VNC build that could configurably send keypresses or EventGhost events, but I think having EventGhost's Keyboard plugin listen to SendInput() like AHK would be a simpler design. Don't you think so?

Only if you can get them to work only against the VNC Server, as mentioned above. Otherwise this seems like the only option.

Thanks for all of your help and sorry for the confusion,
Dustin

No problem.

Brett

[dustobub]

What you want is to trap keypresses, but only if they come from HippoRemote. Is there a way to do this? I thought that is you trapped keypresses, you trapped them all. And you don't want to trap them all, right? Any idea what AHK or Launchy do? You distinguish between SendInput and hardware keypresses, but I thought they looked the same at the point you can trap. Am I missing something?

Well, if this were to be written as a separate plugin from the Keyboard plugin, then it should only trap SendInput() to not interfere with the Keyboard plugin if you chose to use them both. However, I think ideally the keyboard plugin would be adapted to also capture SendInput().

Honestly, I thought that if AHK and Launchy can't distinguish between SendInput() and hardware keypresses, so would EventGhost, but something is obviously different in their implementations. Maybe I will talk with some AHK people to see if they know why they behave differently. I think it's a nice feature that the EventGhost keyboard plugin only listens to actual hardware keypresses, so I think that having a checkbox to configure trapping emulated keypresses would be the best design.

My workaround is most definitely a hack. I didn't realize that a new instance of EventGhost gets spawned everytime I run that command. Any ideas on how to prevent that? I could probably handle this inside AHK, but I don't want to waste much time on this method. Repeat keypresses instantly peg my cpu because it spawns 10 billion EventGhost instances

Thanks and happy Thanksgiving,
Dustin

[dustobub]

Well I got the cpu usage down considerably with some VBscript inside of AHK. I'd love to use python directly, but I found much more information about VBS embedded in AHK than embedded python. I'll post my AHK script later tonight or tomorrow once I iron some kinks out. Again, this is very much a workaround, and I still feel adapting the keyboard plugin is the best solution.

Thanks,
Dustin

[dustobub]

Using the functions provided by ws4ahk.ahk available at http://www.autohotkey.net/~easycom/ I was able to get the cpu usage down to levels on par with the Keyboard plugin. Again and again, it's a hacky workaround, but it works

AutoHotKey code snippet:

Code: Select all

#Include ws4ahk.ahk

WS_Initialize()

EG_Initialize =
(
   Set EG = CreateObject("EventGhost")
)

EG_Unitialize =
(
   Set EG = Nothing
)

VolumeUp = EG.TriggerEvent "VolumeUp"
VolumeDown = EG.TriggerEvent "VolumeDown"

WS_Exec(EG_Initialize)

F10::WS_Exec(Volumeup)
F9::WS_Exec(VolumeDown)

WS_Exec(EG_Uninitialize)

WS_Uninitialize()