The idea of having to wave my hands around in the air to control anything in a precision way seems both inaccurate and tiring, and fingerprints are a real concern if you're actually touching the display surface (obviously not if you're talking about the edges, where you'd use the power button or carry it - I'm not objecting because I'm a clean freak).

Actually you're not waving your air to control anything.

That would be more befitting of a Wii-cam and it would be a bad idea for the sensor to be so sensitive when it is used for such delicate tasks.

Instead it would just be a low sensor that works on the control principle of the 10Gui except your hands are not flat on a pad.

It's far from perfect which is why I don't like the idea either but to repeat the point of my earlier post, if you're going to do finger gestures, it's just much more practical to not alter any major component especially if you're going to increase the size of a keyboard.

The vertical edge of a monitor is great for this because all you need to do is mimic the hand gestures of pointing a finger and manipulate things depending on how many fingers the sensor detects.

Even if we're going by buttons, there is absolutely no way you can touch the screen edge no matter how large your hand is unless the buttons were poorly placed. Just try it now and see how much space you can put your hand on the back of the monitor where the buttons can be placed (with a slight alignment towards the edge so that you can spot the color flashes if there are any)

There is no precedent for such a UI being used for any precision purpose or as the general interface for a normal computer system, whereas the device I suggest is merely a potentially novel combination of existing and proven technologies.

umm... a Wii-cam like CamSpace requires way way way more precision than the monitor sensors I'm talking about: http://www.youtube.com/watch?v=v0srY37kkMw

There are even concept motion sensors now that can mimic your hand movements literally - without an object and at the distance of watching a large television set. (Edit: Here it is. Project Natal: http://www.youtube.com/watch?v=g_txF7iETX0 - also I stand corrected about the lack of an object but still full figure detection at that range is already possible.)

I'm not going to attack your device suggestion besides mentioning that the keyword there is "novel" combination since I want to emphasize the point here that I'm not attacking your idea (much) but merely mentioning the absurdity of some of your claims against my idea.

For example, I would understand if you say that the design was just bad for so and so legitimate tech design reasons but no precedent for such a UI?

Just a wireless mouse can show you the possibility of how possible motion detection is today and did I mention webcams?

Quote from: tomos on October 13, 2009, 02:20 PM

Another thing that I would miss would be the ability to have say two windows one side of the screen and one on the other - but neatly filling the screen - a la GridMove. This hasnt really been considered I think.

The basic idea could be extended so that dropping one window onto another would stack them vertically. Each window in the horizontal strip could then be a vertical strip of windows, which are manipulated in the same manner.

Yes: I had the same thought... though personally, I'd probably go for something more along the lines of Google Earth or SketchUp for moving window views... just 2D with all my fingers seems a little limited

Paul, I'm not entirely sure how to respond. Your concept of touching, or nearly touching the monitor for regular interaction just seems impractical to me. Why? Think about ergonomics: http://ergonomics.about.com/od/office/ss/computer_setup_2.htm Note that it says to place your monitor *at least* 20 inches from yourself. Now my arm is 25-26" to the tips of my fingers. My hand by itself, from wrist to the tip of my long middle finger is about 8 inches. So that means I'd be holding my arm out fully extended trying to manipulate things *all day long*. That's going to quickly get tiring, and develop into some kind of RSI quite soon, I'm sure.

JavaJones, look at your previous criticisms. You never brought this up and instead used such words as "arm waving",  90% too far away and fingerprints on the edge of the screen.

It's disingenuous of you to switch arguments constantly without acknowledging first how silly and mistaken your original arguments were.

Most importantly, impractical is a far cry from silly.

It's even disingenuous to say it would be quickly tiring when part of the reason people get Carpal Tunnel Syndrome is because keyboard positions and mouse positions are not adapted to prevent fatigue.

Ergonomics equals comfort and has very little to do with fatigue.

In fact, if you actually were thinking of that screenshot then you would realize how unergonomic touchpads can be.

Can you imagine how frustrating it would be to accidentally swipe a pad because of the constraints of space if you were to have a pad jutting out of that keyboard or a pad where instead of the shape of the mouse alleviating discomfort from your wrist, you are forced to lie your palm flat on a surface even if we're talking un-even surfaces?

You also forgot the ergonomic factor that if you don't want to use monitor sensors, you just cover it. Don't want to use touchpads? Need to pull it out and replace it with a mouse.

No offense intended but it seems to me that it sounds impractical to you because you are trying to pigeonhole the sensors into something it's not. The 10Gui is not replacing keyboards here. It is replacing the mouse.

Your argument would be the equivalent of saying it is going to be quickly tiring to rest/constantly move your hands on the mouse because you can't rest both of your hands on the keyboard.

Maybe your monitor is closer than mine (and closer than recommended), but if you're conforming to ergonomic guidelines then I don't see how your idea is functional. And certainly no one would want to build a fundamental interaction device for a computer that inherently defies guidelines for ergonomic computer setups.

Any touchpad no matter how superior inherently defies guidelines for ergonomic computer setups.

That's why mostly artist has adapted to tablet pcs while many have not. Without reprioritizing their computer goals, the ergonomics of the tablet pc isn't there.

This holds the same for laptop touchpads and I've seen many insert a mouse because it is so un-ergonomic.

In fact, for the tasks of the 10Gui it is less un-ergonomic to play thumb chopsticks flat on a surface than it is to temporarily point your finger at the edge of the monitor just as it is no more un-ergonomic to push the power button of your monitor off than to whine how every monitor doesn't come with a remote because it is "too far" according to "guidelines for egonomic computer setups".

This may surprise you but the "Natal" video doesn't actually show very precise interaction. Try using that, or even the Wii interface, to precisely select a single word from a paragraph of text. That's the kind of UI interaction I deal with constantly on a daily basis, and anything that is going to replace my PC UI device has to be at least as good as the basic mouse in that regard.

This may also surprise you but the range of the Natal is so far compared to the fingers close to your monitor and we're talking about concept ideas here.

It is again a disingenuous red herring on your part to go from silly... impractical... no precedent for a UI...and now this!

You're a guy suggesting a touch surface to replace a mouse and you're using an example of precisely selecting a single word from a paragraph?!

Again, finger gestures are supposed to replace a "mouse" not a "keyboard".

@Stoic Joker,

It's probably because it's even way more out there compared to Courier and Natal.

Actually, I was more thinking that the command surface would also dynamically change based on the app. When you're running something like Photoshop, the main part of the panel would display Photoshop controls. When you switched apps, it would switch to a 'control panel' for that app.

Standard items like file open/close/save/print/next/previous/etc. could be assigned permanent locations (ex: an icon bank across the top of the command area) among all apps for consistency.

Or a zoom-in/out gesture  - screen-app-window-object?

In many respects (and much as it pains me to say it *choke*) Apple's iPhone incorporates a lot of this already. My GF just upgraded her AT&T cellular plan and got a 3G as part of the deal. Despite my general dislike of Apple for their proprietary closed platform and elitist mindset, even I have to grudgingly admit that the interface design is, for the most part, quite impressive.

But with the way most apps work these days, right now I think the alphanumeric keyboard might actually be in danger of being on the lagging edge of where interfaces are heading.

Keyboards are actually pretty good at making multiway selections. They are superior to touchscreens because the usual choices can be made by muscle memory alone, without having to look. Whatever variety you use (standard keyboard, chord keyboard, or marking menu as with KeystrokeCE) the limit (without shifting) seems to be a 32-way choice, which is also a good limit for the number of options to display at a time.

I'm not sure if this can be done in Windows, or any other OS, but it potentially splits the program from the interface entirely.  All the former has to do is provide lists of functions it makes available, which the interface device displays and selects as it sees fit; and send a list identifier at the appropriate point to tell it which list to switch to.

Hmm. I can see this is going nowhere. We'll see which of our ideas ever sees support, much less realization, but I stand by my previous arguments.

In a funny coincidence I went to Best Buy a few days ago to buy a digital camera for a friend (10% off coupon made it worthwhile over online vendors who were out of stock anyway). While I was there I saw a fairly expensive "edge touch" picture frame where you used sensors on the edge of the screen to control the settings and UI. It was... awful.

- Oshyan

Very well. If you're too prideful to admit that you're jumping through hoops with your reasoning and that you didn't really stood by your arguments and you kept changing them...

Btw. I'd just like to point out that I am not for monitor sensors.

If you had only looked past your blind bias, you would have seen that I preferred a much more developed KeyStrokeCE for the desktop and only mentioned monitor sensors to keep on topic with the 10Gui concept.

Hmm, yes I suppose from your perspective you are faultless in your reasoning and I am foolish. Nevermind that if monitor sensors weren't a focus of your intention, one wonders why you spent so much time and energy arguing in their favor.

No, you have it backwards. You threw out the words silly first. You were the ones who kept changing the argument without even admitting that you were mistaken in the previous one. You are the one accusing me of using your own flawed arguments as some kind of pillar instead of admitting how absurd they were.

Looking back through what you have said, you've described the implementation in multiple ways, and I could as easily perceive that as "changing your argument", just as you have (incorrectly) with mine. All of the major points I made remain valid. The fingerprint issue is something you latched onto because it was easy to argue against as it didn't apply to your concept, but it was hardly a pillar in my argument.

Lol, I never latched on to anything. I'm the one who had to constantly switch arguments while tolerating your manner of thinking and arrogance.

I will say that I found your initial explanation of the idea confusing and that naturally my response was based on that early understanding. Whether the fault was mine or yours, one of misperception, or poor explanation, is irrelevant. The focus of my criticisms of the approach evolved as my understanding of your concept evolved, but it was not evasive and, as I said, all the essential elements of my criticisms remain valid.

To use your own words: "Hmm, yes I suppose from your perspective you are faultless in your reasoning and I am foolish"

I note with interest that no one else has jumped into this particular part of the discussion, whether in favor of you or I. Most likely nobody cares about it. Frankly I'm feeling the same way now. The reason I continue to respond is I find your communications rather rude and patronizing, and I'm not so happy to let that stand. But I suppose the only way to end it is to let it die. So this will be my last response. The world can judge me, and the validity of my points in this thread, as they may.

- Oshyan

Riiight.

To use your own words: "Hmm, yes I suppose from your perspective you are faultless in your reasoning and I am silly"

JavaJones, the reason "I" responded to you in the first place was because it was you who threw out the first insult and was rude and patronizing in your subsequent replies and you continue to be right now in this post.

I suggest instead of pretending your points is the one in need of judgement, you drop the arrogance and look at your personality first. 

@housetier:

It's not an argument of who started it first. That's pretty much clear with this post:

Well, I think Mouser has already expressed some of my concerns. I like the idea of a better multi-touch interface close to hand, rather than the silly idea of trying to actually use your monitor which is A: too far away 90% of the time and B: you don't want fingerprints all over. However I think all the potential of the touch interface is wasted in this concept because it spends too much time and UI commands trying to improve on existing window management solutions when I honestly don't find the existing solutions to be that big a problem. I routinely have 10 or more apps/windows open, and many of these have tabs of their own inside (pspad text editor with tabs, Firefox, Chrome, IE all with tabs, etc.). So for me this concept video is trying to solve a problem I don't have with an intriguing interaction device that is ultimately wasted due to the misdirected UI changes.

- Oshyan

It's more of a rude comeback/reminder to JavaJones' own rude comments as to why we're arguing about monitor sensors.

In that sense, it does fit the "arguments about alternatives" bit. We're just being rude to each other because we're coming off rude to each other.

Back to mobile pcs: how do they manage several windows on a small screen?

Well, I don't own an Iphone or a new Smartphone but maybe by new you also meant to include the older PocketPCs/Palms. (based on your comment about how they manage several windows on a small screen)

In that case, they manage multiple windows through a drop down arrow.

It still depends on the software you have but many of them are designed to work via a drop down arrow or some form of alt tabbing.

In a way, it's no different than cellphones with graphical icons. You press a button. Get a menu. Click on an icon that opens up a menu in full screen (except you can choose to have those applications remain open.)

Quote from: mouser on October 13, 2009, 04:18 PM

i think 40hz's old startrek picture does point the way to the future -- customized input pads tailored to the application you are working on.

Yep:
http://www.artlebedev.com/everything/optimus-tactus/

That's exactly it! Schwing!!! 

(Thx for the link T-Man!)

Quote from: JennyB on October 15, 2009, 06:43 PM

Keyboards are actually pretty good at making multiway selections. They are superior to touchscreens because the usual choices can be made by muscle memory alone, without having to look.

Excellent point. I tend to like hotkeys myself.

The only problem comes when you start supporting multiple applications with widely varying sets of controls and features. You'll see this mostly in music, media, and graphics applications.

Here's an interesting video of what was possible in that direction nearly 15 years ago, and a sensible use of multi-touch support. In either case, the main point is to move away from the mouse having to select both the action and its object.

Eventually you run out of logical key combinations for all the tasks you want to have a key for. Once that happens, you're forced to use arbitrary and non-intuitive ones. A good example is V for PASTE or W for CLOSE in most apps.

Yes, that's one of the reasons I gave up using Dvorak! 

That's why I'm proposing that keycodes not be hardwired to particular keys or gestures. 

The idea comes from ColorForth which uses only 27 keys of a normal keyboard - have a continually updated display of key assignments (not on the keycaps - you should not be looking there anyway) that is generated from lists of currently available commands. The user only sees the command label, not the keycode that it returns. Think of it like a link in the Help index.

Initially, with a new program, no keys are assigned - or perhaps the input device auto-assigns those with familiar labels. Commands are selected by picking with a mouse, or perhaps in FARR style.  When you learn a command and know you are going to use it often, drag a copy off the list and place it on a spot associated with a particular key.

Was IBM's dynamic model somewhat similar to what you had in mind?

From all I know of it (which is just what you have written above) it is very similar, because my idea seems to imply an "every app is a server app" model. It goes like this:

The current list of available actions can be likened to a directory listing. It can be changed by program action, by navigating up or down the heirarchy, or by  selecting a different object. A separate zoom control selects actions suitable for the appropriate level of nested objects, and changes the selection cursor to suit.  So, at the top level you would find actions for logging on and off, creating new tasks, and choosing which screen to focus. Below that, actions relating to individual windows (move, resize, etc), and below that, actions peculiar to the particular program.

On Object Selection (MouseDown)
Controller sends the mouse position. Server returns the object path (The list of all nested objects at that point).

If the actions currently displayed are still in the object path, they remain unchanged and the nested object at that level is selected. So if, for example, you have been resizing windows, a new window can be moved by selecting any point on it. No trying to click on lines, corners and fiddly little handles. Otherwise, show the actions suitable for the new object at the current level.

Optionally, change the action selection.

On MouseUp, the controller sends the current object path and the token for the current action.

The main difference is that I am thinking of the controller as a personal device, perhaps a netbook or a mobile phone, so standard controls are not the issue - no more QWERTY!

I don't really know what a server app specifically is but I don't get why every app should be a server app and why it would improve the system? (aside from creating a uniform set of similar application I guess...)

Anyone have the layman translation?

Seems like it's about rebuilding an OS where as up to now, I was assuming JennyB's concept was just a macro monitor on steroids.