Developing for KEITH or LS

\\

One the server side xtext is used to provide a language server. On the client side the Theia framework is used to communicate with it. This guide is for KIELER developers who want to build functionality in form of language server extensions for KEITH and frontend developers for KEITH that want to know how to communicate with the LS.

6

7

8

9

= Starting the Language Server (LS) =

10

11

This part of the guide covers the server side development for KEITH, the language server (LS). It is advised to have a look at the existing implementation in the language.server plugin.

12

13

=== Register start hook (Eclipse) ===

14

15

Currently the LS is an eclipse application and it has to be started as one. To do this the Manifest.mf in the META-INF folder of your language server plugin has to be changed. In the extensions tab a new extension point for org.eclipse.core.runtime.applications has to be created. This extension points maps to your start class of your language server, which has to be an IApplication that implements a start method.

16

17

[[image:attach:lsstarthook.png]]

18

19

This start class should somehow distinguish between [[connecting via socket>>doc:||anchor="ConnectionviaSocket"]] and [[connecting via stdin/out>>doc:||anchor="Connectionviastdin/stdout"]].

20

21

=== Language Registration ===

22

23

Language that are registered here are always xtext languages.

24

25

==== Register languages that are defined in the semantic ====

26

27

Since we are in the semantics repository we can use java ServiceLoader to add new ILSSetups, which register a language.

interface ILSSetup {

def Injector doLSSetup()

32

}

33

34

class SCTXLSSetup implements ILSSetup {

35

override doLSSetup() {

36

return SCTXIdeSetup.doSetup()

}

}

A language that wants to be included in the LS can implement this interface. Registering SCTXLSSetup via [[ServiceLoader>>doc:KIELER.HowtouseServiceLoader]] allows to register all available languages like this:

42

43

44

for (contribution: KielerServiceLoader.load(ILSSetupContribution)) {

45

contribution.LSSetup.doLSSetup()

}

==== Register that are defined outside of the semantic ====

50

51

Have a look at one of the LSSetups defined in the semantic.

=== Bindings ===

Bindings for the injector created by createLSModules and why they are needed:

56

57

* the KeithServerModule bind all standard stuff

58

* if we start via stdin/out, the ServerLauncher has to be bound to the LanguageServerLauncher

59

* we have to bind a ResourceRegistry and our own WorkspaceConfigFactory that allows to open a folder without xtext getting involved

60

* all bindings for the KGraphDiagramModule and KGraphDiagramServerModule are done to have a working diagram server

61

62

=== Starting and connecting ===

63

64

The following things are done via the LanguageServer class on startup of the LS:

65

66

* port and host argument are read. This decides whether the LS starts via socket or stdin/out

67

* socket

68

** call bindAndRegisterLanguages (this loads all languages, languages defined in the pragmatic have to be added manually, all other ones are added via the KielerServiceLoader (see ILSSetup))

69

** injector is created via createLSModules

70

** a new socket is opened, buildAndStartLS is called (gets all ILanguageServerContribution via KielerServiceLoader and starts LS)

71

* stdin/out

72

** main of LanguageServerLauncher is called

73

** this calls bindAndRegisterLanguages

74

** launch of ServerLauncher is called with injector created by createLSModules

75

** this calls the start method which calls buildAndStartLS

\\

== Connection via socket ==

81

82

You have to specify a port and an optional host as VM arguments to start the LS via socket.

83

84

Currently the LS is an eclipse application, therefore you create a new Eclipse Application run configuration in eclipse.

85

86

In the {{code language="none"}}Main{{/code}} tab, section {{code language="none"}}product to run{{/code}} select Run an application and select {{code language="none"}}de.cau.cs.kieler.language.server.LanguageServer{{/code}}.

87

88

In the {{code language="none"}}arguments{{/code}} tab, section {{code language="none"}}VM arguments{{/code}} add{{code language="none"}} -Dhost=localhost -Dport=5007{{/code}} to the arguments. You might want to specify {{code language="none"}}Xmx{{/code}} here too, if you plan to use bigger models (such as the railway controller).

89

90

Running this eclipse application requires a Theia client that tries to connect via socket to an LS running on port 5007.

91

92

== Connection via stdin/stdout ==

93

94

Is only relevant for the product. Requeires to start to LS without an host or port Vm argument. Cannot be debugged.

95

96

= Developing a LS extension =

97

98

We use java ServiceLoader to register stuff. Here is a small example how a LanguageServerExtension is registered via a ServiceLoader and how it is used:

\\

== Register LanguageServerExtensions (ServiceLoader Example) ==

103

104

This is a LanguageServerExtension. It has to be used in the de.cau.cs.kieler.language.server plugin. Since the language-server-plugin should not have dependencies to all plugins that define a language server extension dependency inversion is used to prevent that. A ServiceLoader via dependency inversion does exactly that.

105

106

Here is such an example extension, the KiCoolLanguageServerExtension:

107

108

109

package de.cau.cs.kieler.kicool.ide.language.server

/**

* @author really fancy name

*

*/

@Singleton

class KiCoolLanguageServerExtension implements ILanguageServerExtension, CommandExtension, ILanguageClientProvider {

118

// fancy extension stuff

119

120

KeithLanguageClient client

121

// A language server extension must implement the initialize method,

122

// it is however only called if the extension is registered via a language.

123

// This should never be the case, so this is never called.

124

override initialize(ILanguageServerAccess access) {

125

this.languageServerAccess = access

126

}

127

128

// implement ILanguageClientProvider

129

override setLanguageClient(LanguageClient client) {

130

this.client = client as KeithLanguageClient

131

}

132

133

// implement ILanguageClientProvider

134

override getLanguageClient() {

return this.client

}

}

The CommandExtension defines all commands (requests or notifications) that are send from client to server. An example how this looks like can be seen in the code snippet Example CommandExtension is an example how to [[define a server side extension interface.>>doc:||anchor="Registeranextension(onserverside)"]]

142

143

The ILanguageClientProvider should be implemented by an extension that plans to send [[messages from the server to the client>>doc:||anchor="ServerClientcommunicationinterface"]].

144

145

This language server extension is provided by a corresponding contribution, which is later used to access it:

146

147

148

package de.cau.cs.kieler.kicool.ide.language.server

149

150

import com.google.inject.Injector

151

import de.cau.cs.kieler.language.server.ILanguageServerContribution

152

153

/**

154

* @author really fancy name

155

*

156

*/

157

class KiCoolLanguageServerContribution implements ILanguageServerContribution {

158

159

override getLanguageServerExtension(Injector injector) {

160

return injector.getInstance(KiCoolLanguageServerExtension)

}

}

Create a file called de.cau.cs.kieler.language.server.ILanguageServerContribution in <plugin>/META-INF/services/ (in this example this is de.cau.cs.kieler.kicool.ide). The name of the file refers to the contribution interface that should be used to provide the contribution. The content of the file is the following:

166

167

168

de.cau.cs.kieler.kicool.ide.language.server.KiCoolLanguageServerContribution

169

170

171

This is the fully qualified name of the contribution written earlier.

172

173

The language server uses all LanguageServerExtensions like this:

174

175

176

var iLanguageServerExtensions = <Object>newArrayList(languageServer) // list of all language server extensions

177

for (lse : KielerServiceLoader.load(ILanguageServerContribution)) { // dynamically load all contributions to add LS extensions

178

iLanguageServerExtensions.add(lse.getLanguageServerExtension(injector))

}

The resulting list of implementions is used to add the extensions to the language server.

183

184

185

The interfaces used for dynamic registration are in the semantics repository. If you define a pragmatics LS extension you have to statically add these extensions to your list.

186

187

188

== Register an extension (on server side) ==

189

190

See example above for ServiceLoader and initial stuff.

191

192

What is still missing are the contents of the CommandExtension implemented by the KiCoolLanguageServerExtension. This is an interface defining all additional commands. The CommandExtension looks like this.

193

194

195

package de.cau.cs.kieler.kicool.ide.language.server

196

197

import java.util.concurrent.CompletableFuture

198

import org.eclipse.lsp4j.jsonrpc.services.JsonRequest

199

import org.eclipse.lsp4j.jsonrpc.services.JsonSegment

200

201

/**

202

* Interface to the LSP extension commands

203

*

204

* @author really fancy name

205

*

206

*/

207

@JsonSegment('keith/kicool')

208

interface CommandExtension {

209

210

/**

211

* Compiles file given by uri with compilationsystem given by command.

212

*/

213

@JsonRequest('compile')

214

def CompletableFuture<CompilationResults> compile(String uri, String clientId, String command, boolean inplace);

215

216

/**

217

* Build diagram for snapshot with id index for file given by uri. Only works, if the file was already compiled.

218

*/

219

@JsonRequest('show')

220

def CompletableFuture<String> show(String uri, String clientId, int index)

221

222

/**

223

* Returns all compilation systems which are applicable for the file at given uri.

224

*

225

* @param uri URI as string to get compilation systems for

226

* @param filter boolean indicating whether compilation systems should be filtered

227

*/

228

@JsonRequest('get-systems')

229

def CompletableFuture<Object> getSystems(String uri, boolean filterSystems)

}

This defines three json-rpc commands: "keith/kicool/compile", "keith/kicool/show", "keith/kicool/get-systems". These are implemented in KiCoolLanguageServerExtension.

\\

== Server Client communication interface ==

239

240

Not only messages from client to server but rather messages from server client might be needed.

241

242

Messages that can be send from server to client are defined in the KeithLanguageClient:

/**

* LanguageClient that implements additional methods necessary for server client communication in KEITH.

247

*

248

* @author really fancy name

249

*

250

*/

251

@JsonSegment("keith")

252

interface KeithLanguageClient extends LanguageClient {

253

254

@JsonNotification("kicool/compile")

255

def void compile(Object results, String uri, boolean finished);

256

257

@JsonNotification("kicool/cancel-compilation")

258

def void cancelCompilation(boolean success);

259

260

// Not only notifications, but also server client requests should be possible, but currently there is no use case for that.

}

These messages can be caught on the client side by defining the message that is caught like this:

265

266

267

export const snapshotDescriptionMessageType = new NotificationType<CodeContainer, void>('keith/kicool/compile');

268

269

270

This message type is bound to a method that should be called whenever the client receives such a message.

271

272

273

const lClient: ILanguageClient = await this.client.languageClient

274

lClient.onNotification(snapshotDescriptionMessageType, this.handleNewSnapshotDescriptions.bind(this))

275

276

277

The method should receive all parameters specific in the KeithLanguageClient interface on the serevr side.

278

279

Such a notification from server to client is send like this:

future.thenAccept([

// client is the KeithLanguageClient registered in a LanguageServerExtension that implements a ILanguageClientProvider

284

// compile is the command defined in the KeithLanguageClientInterface

285

client.compile(new CompilationResults(this.snapshotMap.get(uri)), uri, finished)

])

== Register and calling an extension (on client side) ==

290

291

Language server extension do not have to be registered on the client side. It is just called.

292

293

You can send a request or a notification to the language server like this:

294

295

296

const lclient = await this.client.languageClient

297

const snapshotsDescriptions: CodeContainer = await lclient.sendRequest("keith/kicool/compile", [uri, KeithDiagramManager.DIAGRAM_TYPE + '_sprotty', command,

298

this.compilerWidget.compileInplace]) as CodeContainer

299

// or via a thenable

300

client.languageClient.then(lClient => {

301

lClient.sendRequest("keith/kicool/compile").then((snapshotsDescriptions: CodeContainer) => {

302

// very important stuff

303

}

304

// await is preferred, since it is shorter.

305

306

307

In this example client is an instance of a language client. It is usually injected like this:

308

309

{{code title="Client side LanguageClientContribution injection"}}

310

@inject(KeithLanguageClientContribution) public readonly client: KeithLanguageClientContribution

311

constructor(

312

// other injected classes that are relevant for the constructor

) {

// constructor stuff

}

== How to make a new package for KEITH ==

319

320

Clone the [[KEITH repository>>url:https://git.rtsys.informatik.uni-kiel.de/projects/KIELER/repos/keith/browse||shape="rect"]].

321

322

Open the keith folder in VSCode. You are know in the keith directory in VSCode (see picture an the right).

323

324

Create a new folder called keith-<your extension name>.

325

326

Copy a package.json, a tslint.json, a tsconfig.json, and a src folder into the folder.

327

328

Add keith-<your extension name> to workspaces in the top level package.json.

329

330

Add "keith-<your extension name>(% style="color: rgb(0,0,0);" %)": (%%)"<your-version (e.g. 0.1.0)>"(% style="color: rgb(0,0,0);" %) to the dependencies in the top level package.json and the product package.json files (e.g. the package.json in keith-app).

331

332

=== What is in the src directory? ===

333

334

The source directory has three optional subfolders.

335

336

* node: Holds all backend related classes. This does currently only exist in the [[keith-language package>>url:https://git.rtsys.informatik.uni-kiel.de/projects/KIELER/repos/keith/browse/keith-language||shape="rect"]].

337

* common: Holds general helper methods, string constants and some data classes

338

* browser: Holds all widgets, contribution for commands, menus, and widgets, and the frontend-extension.

339

340

==== The frontend-extension ====

341

342

This binds all necessary classes. Look at existing frontend extension in [[KEITH>>url:https://git.rtsys.informatik.uni-kiel.de/projects/KIELER/repos/keith/browse||shape="rect"]] or [[Theia>>url:https://github.com/theia-ide/theia/tree/master/packages||shape="rect"]] to see how this is done.

343

344

==== More examples for stuff ====

345

346

See [[Theia examples>>url:https://www.theia-ide.org/doc/Commands_Keybindings.html||shape="rect"]].

347

348

[[image:attach:Screenshot from 2019-07-09 12-48-05.png]]

349

350

== How to write a widget ==

351

352

There are different kinds of widgets that are commonly used in [[KEITH>>url:https://git.rtsys.informatik.uni-kiel.de/projects/KIELER/repos/keith/browse||shape="rect"]] or in existing [[Theia packages>>url:https://github.com/theia-ide/theia/tree/master/packages||shape="rect"]].

353

354

* BaseWidget: Very basic

355

* ReactWidget: A render method has to be implemented that redraws the widget on demand. Additionally several on* event methods can beimplemented.

356

* TreeWidget: Extends the ReactWidget and draws the contents of the widget in a tree view.

357

358

If a widget has a state it should implement the StatefulWidget interface, which allows to imlement a store and restore method.

359

360

Look at examples in KEITH or Theia to see how this is done.

361

362

== How to make a new module for sprotty (see actionModule, ...) ==

WIP

= How to use (Kieler)ServiceLoader =

367

368

Classes that are provided via a ServiceLoader have a contribution that provides them. This contributions share a common interface lets call it {{code language="none"}}IStuffContribution{{/code}}. A specific StuffContribution (it provides stuff) is the ImportantStuffContribution.

369

370

371

// defined in some common package

372

interface IStuffContribution {

373

374

abstract def IStuff getStuff()

375

}

376

// defined in the package that holds ImportantStuff

377

class ImportantStuffContribution implements IStuffContribution {

378

override getStuff() {

379

return new ImportantStuff()

}

}

The ServiceLoader has to know that some plugin provides an implementation for IStuffContribution. Therefore, a folder named services is added next to the corresponding MANIFEST.MF.

385

386

In the services folder a file named the same as the fully qualified name of the implemented interface is added. Here this one is called {{code language="none"}}de.cau.cs.kieler.basic.package.IStuffContribution{{/code}}.

387

388

The file holds the fully qualified names of all implementations of this interface:

389

390

391

de.cau.cs.kieler.importatn.package.ImportantStuffContribution

392

393

394

Now you can access implementation of IStuff as follows:

395