Open Source Mobile Communications: Issues 2019-08-05T15:57:19Z Open Source Mobile Communications Redmine Cellular Network Infrastructure - Bug #4141 (New): No osmux specific code in place during codec n... 2019-08-05T15:57:19Z pespin <p>Current code in OsmoBSC and OsmoMSC doesn't take into account that Osmux is to be used when codec negotiation goes one. That means that supported codec list needs to be correctly configured manually in VTY in order to avoid selecting incorrect codecs (like non-AMR). Moreover if an MS doesn't support AMR, since we don't have transcoding yet, the call cannot take place.</p> <p>Different scenarios need to be listed here and think about what we want to do in each case.</p> Cellular Network Infrastructure - Bug #4131 (Stalled): osmo-gsm-manuals: Use leveloffset attribut... 2019-07-26T13:04:49Z pespin <p>Right now, all documents under osmo-gsm-manuals.git/commons/ start with title level 1 (==). However, if one wishes to add a document with level 2 (===) so it can also be included in other repository document, it will make osmo-gsm-manuals.git "make check" fail due to osmo-gsm-manuals/tests/<br /><pre> echo "include::$${chapter}[]" &gt;&gt; $@; \ </pre></p> <p>Which includes stuff under a test document with level 0 (=). If a document with level different than 1 (==) is added, then it will fail on some versions (like jenkins):<br /><pre> "asciidoc: WARNING: mgcp_extension_osmux.adoc: line 2: section title out of sequence: expected level 1, got level 2" </pre></p> <p>See for instance commit osmo-gsm-manuals.git 061cca4d7345bc4f496324d0b4d30bc51e1f8d23, which had to be fixed up later.</p> <p>The solution here is to use "leveloffset" attribute. To make our life easier in the future, we need to move all documents under common/ to start with level 0 (=), and then use "leveloffset" on all includes in all other repositories using them. This way same document can easily be added on different levels on different documents.</p> <p>Important! It seems this syntax doesn't work for me:<br /><pre> include::chapter2.adoc[leveloffset=+1] </pre></p> <p>However, this does and it's basically the same:<br /><pre> :leveloffset: 1 include::chapter1.adoc[] :leveloffset: 0 </pre></p> <p>Related information:<br /><a class="external" href=""></a><br /><a class="external" href=""></a><br /><a class="external" href=""></a></p> Cellular Network Infrastructure - Bug #4108 (New): osmo-ctrl2cgi, osmo-trap2cgi: missing default ... 2019-07-16T12:30:11Z osmith <p>Default configs for osmo-ctrl2cgi and osmo-trap2cgi do not get installed in the debian packages. Therefore the systemd services don't work out of the box.</p> <p>Both programs are part of osmo-python-tests.git.</p> <p>I'm setting this to low priority as I think that the programs are not widely used.</p> Cellular Network Infrastructure - Feature #4107 (New): Start systemd services as non-root user 2019-07-15T06:56:35Z osmith <p><a class="user active" href="/users/7">laforge</a> wrote in <a href="" class="external">OS#3369</a>:</p> <blockquote> <p>Ideally, as far as possible, we should start them as non-root user<br />(which may require changes to our systemd service files, etc. in the<br />individual git repos - but that is fine!). Starting them as non-root<br />will also means that any writes to unintended directories like '/'will<br />be discovered as they then would make the program start fail.</p> </blockquote> Cellular Network Infrastructure - Feature #4091 (New): Osmux: Make sure all handover types work f... 2019-07-05T14:10:17Z pespin <p>It has never been tried and it could be it doesn't work.</p> Let's add some TTCN3 tests to make sure we support different scenarios.<br />For instance: <ul> <li>intra-bsc handover between 2 BTS while BSC<->MSC is using Osmux</li> <li>inter-bsc handover between 1 BSC with osmux enabled and 1 BSC with osmux disabled while a call is ongoing. Test both handover directions (A->B, B->A).</li> <li>inter-msc handover between 2 MSC, one MSC having a BSC with Osmux enabled and another MSC having a BSC with Osmux disabled. Test both handover directions (A->B, B->A).</li> </ul> Cellular Network Infrastructure - Feature #4073 (Stalled): try to get counter and vty reference i... 2019-06-21T14:55:46Z laforge <p>The existing approach of extracting counter and vty information at runtime has various disadvantages. We'd like to explore options to generate this information from the source code. Whether that's using static code analysis, an existing C parser of some form, clang, ... doesn't really matter.</p> <p>The goal is to generate the same textual (asciidoc for counters, xml for VTY) format as the existing 'runtime' approach.</p> Cellular Network Infrastructure - Feature #4044 (Stalled): regenerate vty reference during releas... 2019-06-04T21:34:48Z laforge <p>now that the manuals are part of the respective program git repositories, we should somehow make sure that every time we tag a new release, the VTY reference XML is updated, ensuring [at least] the tagged versions have corresponding vty reference documentation.</p> <p>I know it may not be trivial to completely automatize it. So for now I think I would be happy of the release helper prints a reminder, possibly requiring a manual --override if the vty reference xml didn't change compared to the last release? I'm open for better ideas, but at the very least a simple reminder should be present.</p> Cellular Network Infrastructure - Feature #4040 (New): Use "COPY --from" in Dockerfiles to avoid ... 2019-06-04T09:48:58Z laforge <p>Some of our Dockerfiles clone a lot of git repositories (e.g. those for the ttcn3 tests), and due to the sequential nature of docker cache invalidation, a change in osmo-ttcn3-hacks.git not only means we will re-clone that git repository from scratch, but we also will clone all of the 'deps' repositories from scratch.</p> <p>If we had a docker container hosting pre-created clones of those repositories, we could use the "COPY --from" syntax in our Dockerfile, copying those initial clones of the git repositories over. We'd then only have to pull whatever changes (if any) over the network.</p> <p>See <a class="external" href=""></a> for a description of this feature towards the bottom.</p> This would <ul> <li>significantly reduce the network traffic on the buildhost during re-generation of images</li> <li>reduce the load to the respective git servers</li> <li>speed up the build process</li> </ul> <p>The container hosting the "pre cloned git repos" could then be updated only once per week or even once per month. It should <strong>not</strong> be rebuilt too often, as otherwise it will have to be pulled all the time, avoiding the benefits stated above and just move traffic from git to docker.</p> Cellular Network Infrastructure - Feature #4006 (Stalled): TRX protocol: wind of change 2019-05-16T19:54:33Z fixeria <p>We are using TRX protocol in <a class="wiki-page" href="">OsmoBTS</a> in order to "speak" with transceiver (e.g. <a class="wiki-page" href="">OsmoTRX</a>, FakeTRX). Basically it defines three interfaces: clock for TDMA frame clock indications, control (we agreed to call it TRXC) for transceiver management, and data (TRXD) for exchanging Rx/Tx bursts. It was first introduced in <a href="" class="external">OpenBTS</a> project, from which we forked <a class="wiki-page" href="">OsmoTRX</a>. For more information, please see: <a class="external" href=""></a>.</p> <p>The protocol is being used for years, and it still serves us for good. However, it is extremely inflexible. For example, there is no way to send more information about received bursts on TRXD interface, than the fixed-size header already has (TDMA frame number, timeslot number, RSSI, ToA256). On TRXC interface, which is working over UDP as the other ones, there is no way to distinguish command retransmission, which may happen due to timeout, from a regular command. There is no way to notify the L1 (i.e. <a class="wiki-page" href="">OsmoBTS</a>) about some events (e.g. device has been disconnected), no way to indicate transceiver version, and so on...</p> <p>During <a class="wiki-page" href="">OsmoDevCon2019</a> (and before too) we have been discussing the idea of introducing the new version of TRX protocol, which would allow us to solve the mentioned problems. In order to keep backwards compatibility, both L1 and transceiver would initially use the old TRX protocol, while the new version can be optionally enabled by sending a command on TRXC interface. If the transceiver does support the new protocol, it would acknowledge the command. Otherwise, the command is rejected, so L1 would continue to work in "backwards compatibility" mode.</p> <p>Finally, we need to write a proper protocol description like we already have for GSUP in <a class="external" href=""></a>. But before starting to work on this, let's create some kind of a wish list - what would you like to see in the new version of TRX protocol?</p> Cellular Network Infrastructure - Feature #3982 (New): set dependency hashes on... 2019-05-07T13:48:30Z osmith <p>Right now, (from osmo-ci.git) always uses the versions of the dependencies that are specified in of the Osmocom project that is being tested.</p> <p><a class="user active" href="/users/30187">pespin</a> was just using the script and wanted to check if a project builds against a specific commit of a dependency. He said, it would be nice to have something like:</p> <pre> ./ -b osmo-msc:7231edb7321a238851387479df0ee16d6c936de0[libosmocore:aa98c481fa7888e2bcd9d5c9ae1993744ee47f33] </pre> <p>(so after the project:hash, add [] with a list of dependencies)</p> Cellular Network Infrastructure - Bug #3753 (New): Update .deb to use dh_missing 2019-01-09T13:36:05Z msuraev <p>While building .deb for libosmocore following warning was observed:<br /><pre> sed -i "/dependency_libs/ s/'.*'/''/" `find . -name '*.la'` dh_install dh_install: Please use dh_missing --list-missing/--fail-missing instead dh_install: This feature will be removed in compat 12. </pre></p> <p>We should follow this recommendation and update our packaging before compat 12 becomes mainstream. We should also check other Osmocom .deb packages because they might have similar issues.</p> Cellular Network Infrastructure - Bug #3725 (Stalled): Jenkins isn't using the credentials store ... 2018-12-12T13:57:52Z osmith <p>The jenkins jobs generated by osmo-ci.git/jobs/master-buils.yml need to be able to upload artifacts to (generated PDF manuals, generated firmware files).<br />Right now, we have the SSH keys of each build slave configured in the authorized_keys of, but that does obviously not scale.</p> <p>laforge wrote in <a class="issue tracker-1 status-3 priority-3 priority-high3 closed" title="Bug: Jenkins can&#39;t upload to (Resolved)" href="">#3720</a>:</p> <blockquote> <p>The proper solution is to use the jenkins server credentials store, which will hold the private key and provision it to the client via ssh-agent. That way the client can upload to the server, and no per-slave configuration is required on the ftp server.</p> </blockquote> <p>Some of the scripts that require the SSH keys are running in docker. Right now we are mounting ~/.ssh in the containers, using the ssh-agent should be possible when mounting the socket and passing the environment variable:</p> <pre> docker run --rm -it --name container_name \ -v $(dirname $SSH_AUTH_SOCK):$(dirname $SSH_AUTH_SOCK) \ -e SSH_AUTH_SOCK=$SSH_AUTH_SOCK my_image </pre> <p>(<a href="" class="external">found here</a>)</p> Cellular Network Infrastructure - Feature #3699 (Stalled): merge Debian patches 2018-11-19T13:41:54Z msuraev <p>There're numerous patches from Debian developers available via:<br /><a class="external" href=""></a><br />individual package links example:<br /><a class="external" href=""></a><br />Relevant package version could be obtained from the first link.</p> <p>Some of those patches are Debian-specific while others are generally relevant fixes. We should go over all packaged projects and submit all relevant patches to gerrit for review.</p> <p>Note: Debian seems to have the most recent versions packaged compared to other repositories according to repology - see for example<br /><a class="external" href=""></a></p> Cellular Network Infrastructure - Feature #3670 (New): TTCN-3 tests for statsd exporter code 2018-10-24T18:46:26Z laforge Cellular Network Infrastructure - Bug #3660 (Stalled): TTCN3 Jenkins jobs leave Docker containers... 2018-10-17T09:26:36Z osmith <p>The scripts start multiple containers and stops them at the end of the script. When aborting a job with the web interface, the script gets killed and can not do the clean up part. <a href="" class="external">Jenkins kills all child processes</a>, but not the containers as they get spawned by the docker daemon.</p> Ideas to resolve this: <ul> <li>open a bug report at Jenkins to not kill processes with -9 in a bug report, then wait for a stable release with the fix, then implement clean up on receiving SIGABRT in</li> <li>try out the <a href="" class="external">PostBuildScript Plugin</a> (or <a href="" class="external">Post build task</a>, that <a href="" class="external">should work</a> for sure), see if they allow executing code on abort of the job</li> <li>try to attach the environment variable that jenkins is using to decide which processes will be killed (see link above) to the processes in the Docker containers</li> </ul> Cellular Modem Information - Feature #3658 (New): Create a "osmocom-bb-host-latest" docker container 2018-10-16T14:11:32Z osmith <p>The ttcn3-bts-test-latest jenkins job should use the "latest" version of everything that is used for the test.<br />But since there is no "latest" tag in the osmocom-bb.git repository, we can't build a osmocom-bb-host-latest docker image.</p> <p>The current workaround is using osmocom-bb-host-master for ttcn3-bts-test-latest.</p> Cellular Network Infrastructure - Bug #3417 (New): show asciidoc counters does not show all the c... 2018-07-25T07:01:29Z daniel <p>Since show asciidoc counters only iterates through the allocated rate_ctr/stat_item groups it does not necessarily show all the counters that can be available.</p> <p>For example the sgsn only creates (and reports) bssgp:bss_ctx counters if there is actually a bssgp configured in the sgsn. While you could argue that this is just a misconfiguration the issue remains with counters allocated for each mm or pdp context. As long as no phones are GMM attached the command show asciidoc counters does not know about any statistics for mm or pdp context.</p> <p>We could add commands to add the rate_ctr_group_desc ans osmo_stat_item_group_desc to another list which keeps track of available counters and traverse that for show asciidoc stats as we don't need the actual allocations for counter values that the *_group_alloc() functions do. Group and counter description needed for documentation are all available from the *_group_desc structs.</p> <p>This would mean that we need to register every counter group independently from the *_group_alloc() calls happening right now, so the documentation knows about them - but as a side effect we could also have a function to allocate a rate_ctr/stat_item group by name (instead of struct pointer). Not sure how useful that would be.</p> Cellular Network Infrastructure - Bug #3192 (New): meas_vis: use the lchan identity as primary ke... 2018-04-21T19:21:40Z neels <p>While re-adding meas_feed to osmo-bsc.git, I noticed that the meas_vis and meas_web tools appear to use the lchan's IMSI as primary key to visualizing active lchans. They should use the bts,trx,ts,ss number instead.</p> <p>See also: <a class="issue tracker-2 status-3 priority-2 priority-default closed" title="Feature: resurrect meas_feed (Resolved)" href="">#2968</a> <a class="issue tracker-2 status-1 priority-1 priority-lowest" title="Feature: obtain and store subscriber identity (New)" href="">#2969</a></p> <p>meas_vis is currently still in openbsc.git, it should probably move over to osmo-bsc <a class="issue tracker-2 status-6 priority-2 priority-default closed" title="Feature: move meas_vis and meas_json over to osmo-bsc.git (Rejected)" href="">#3191</a>.<br />meas_web is external, but it could make sense to send a pull request.</p> Cellular Network Infrastructure - Bug #3127 (New): update the [[Osmocom Network In The Box]] wiki... 2018-03-30T14:45:12Z neels <p><a class="wiki-page" href="">Osmocom Network In The Box</a></p> <ul> <li>MGW endpoint management has changed, there is no need to provide the endpoint range with the MGCP clients anymore. Clarify with dexter and edit away the parts concerned with that.</li> <li>GGSN routing: at the time of writing and until now I'm not a routing/iptables guru. I've since added a '-s' to my iptables rules to only nat for phones' IP addresses; so that whole topic might be good to review in this wiki page / the GGSN manual. It's not compulsory that we say much about that topic in our docs, but at least make sure we don't actively mislead with my half-knowledge. (BTW, I'm still seeing my nat thwarted when the docker service is running on my machine)</li> <li>replay the howto 1:1 and see if everything still works, update the attached tar of config files</li> </ul> Cellular Network Infrastructure - Bug #2839 (New): statistics split betwteen 2G and 3G 2018-01-18T20:05:59Z laforge <p>many of our statistics/countrs currnetly don't distinguish between 2G and 3G bearer. This means we cannot know how many 2G vs 3G LU we have received in a given period of time, for example.</p> Cellular Network Infrastructure - Feature #2623 (New): SCCP/M3UA: detect restart of osmo-msc and ... 2017-11-07T23:25:36Z neels <p>Connecting osmo-bsc and osmo-hnbgw to the MSC and SGSN via an OsmoSTP instance, it is currently not possible to detect that the MSC or SGSN has restarted.</p> <p>Scenario: using a sysmoBTS as a NITB, change MSC config, restart MSC -- now osmo-bsc happily continues to run and does not even notice that it is an entirely new MSC instance running in the core net now.</p> <p>In the old days, the SCCPlite link would go down, but since now OsmoSTP is in-between and has no concept of who depends on who, no-one is notifying BSC or HNBGW that MSC or SGSN have gone down. Find out how this is intended to be solved if at all, and devise a way how osmo-bsc will restart and/or reconnect to a new MSC instance, and so forth.</p> Cellular Network Infrastructure - Feature #2558 (Stalled): Scripts to manage thousands of "mobile... 2017-10-06T14:53:38Z laforge <p>The goal here is to be able to run a network with hundreds of BTSs, several hundreds of TRXs and thousands of MSs from a single command, in order to perform load testing against OsmoBSC.</p> <p>The scripts should make sure all related processes are started reliably, their termination is recorded, and somehow their result / error / logging is aggregated and reported.</p> Cellular Network Infrastructure - Feature #2554 (New): Reference/Demo setup + documentation for u... 2017-10-06T14:39:27Z laforge Cellular Network Infrastructure - Feature #2537 (Feedback): package wireshark with osmocom patches 2017-10-05T07:58:25Z laforge We should have some jenkins-ci job that <ul> <li>packages our wireshark as packages in OBS as part of the feeds</li> <li>possibly even re-bases our non-mainline patches before each build</li> </ul> <p>This is particularly important for the AMR voice playback, as wireshark has refused to include this upstream. We can also include any other not-yet-mainline code that we have at osmocom, like osmux statistics, Jacob's EGPRS patch, Ericsson RSL support, ...</p> <p>It would probably be best if both the package as well as the file names are modified to not conflict with the distribution-standard wireshark package that mostly everyone has installed. Let's hope this is not too hard to do. Alternatively we could install into something like an /opt/osmocom PREFIX. A bit ugly, but then it would avoid having to patch the wireshark source?</p> <p>Please coordinate with Daniel on this.</p> Cellular Network Infrastructure - Feature #2385 (New): validate debian rules/control as part of j... 2017-07-21T13:01:41Z laforge <p>Right now one can make changes to the source code that will break the debian package builds.</p> <p>It would be nice if the jenkins build testing for +Verified in gerrit would trigger a dpkg-buildpackage of the related code. This is probably a good case for a docker container that has all known/shared required packages installed?</p> Cellular Network Infrastructure - Feature #1975 (New): Redesigned configuration management / MIB 2017-03-09T01:05:50Z laforge <p>Using zebra/quagga VTY code was a great idea back then and has served us nicely in all those years. I like the general style of the command-line based interface with its various nodes, the strict syntax checking, the tab completion and the interactive context-sensivive help.</p> <p>However, it feels a bit 20ieth-century-ish to have to manually write code to parse and to save the respective values. This is not a productive way to spend our development resources, and it is error prone. The "save" can be forgotten, resulting in non-saveable config parameters. The save can store values that the "parse" function will not be able to read again, etc.</p> <p>Furthermore, the VTY is inherently a human user interface, not intended for programmatic consumption. For programmatic access, we have developed the control interface. In reality though, only 1% of the parameters available in the VTY are exported via the control interface. And rather than adding all the missing bits and pieces with hand-crafted code to the control interface, we should have a generic way to define a new configuration value, and that value should then be automatically parse- and storable via VTY as well as a programmatic interface.</p> <p>The next "problem' is that the current telnet connections live within the process of the application. This means that a user can effectively "stall" the main application by performing I/O intensive operation on the VTY, or even on as many VTY/telnet sessions as he wants. There shouldn't be any need for this, at least not for something as mundane as performing configuration changes. Of course the VTY has different use cases such as runtime introspection of system state as well as logging. But still.</p> <p>Yet another concern is "VTY and telnet port proliferation". Particularly with the conversion from NITB to BSC+MSC+HLR, we are yet again getting more network elements with their own telnet ports. Remembering the port numbers is clumsy. It would be more convenient if a single command line interface could provide access to the configuration and the state of multiple different processes / network elements.</p> <p>Last, but not least, the current implementation is fixed to telnet, without any form of authentication, and without a path to migrate e.g. to something like SSL or SSH. I don't think it is a major concern (you can always SSH to the system and then telnet locally).</p> <p>So what I had in mind for quite some time (actually since netconf 1.2 about one year ago), is to have some kind of an external "VTY/MIB daemon" (or even separate daemons for each) which maintains a hierarchical database of configuration values. The MIB deamon simply offers an API (via client library) to GET or SET the individual values, or to NOTIFY an application about a changed value. This API is both used by the actual Osmocom programs to obtain their configuration (and obtain updates to it during runtime), as well as by the "VTY daemon" providing interactive shell access to it. Finally, other external applications could use the same interface/client library to do the same. A proxy to SNMP or REST interfaces can be imagined, for even more interfacing with the outside world.</p> <p>What I don't like about many existing MIBs I've used (as a sysadmin/user, not a developer) is their simple type system. You can often specify any random value to such a MIB, even one that is completely useless. A simple INT or STRING type is not sufficient, we really want the ability to have ENUM types, to have integers with ranges, etc. - just like we have in the VTY.</p> <p>Also, the MIBs I've used typically are nothing more than a hierarchical key-value store. They do not contain the information required for interactive, context-sensitive help needed for the "VTY" feel :( This is btw also the problem I have with OpenWRT's uci: It just has keys and values, with no way to constrain those values to something that's reasonable to the given program/context that is being configured, and there's no help or other information associated with those keys and values.</p> <p>So what I would like to see in this context, is some way to have a machine-parseable description of a "MIB/config item", together with syntax, ranges, enum values, help texts, etc (ideally in the C source code, possibly in comments?) which is used by some kind of MIB compiler to build up the hierarchical structure of the MIB and all of its keys as well as their permitted values and syntax reference. This information is then available to the VTY/telnet connections, irrespective of whether a given application [using those values] is running at all.</p> <p>Once an application starts, it can query for the configuration values it is interested in and populate its internal data structures from it. Ideally, one would even be able to generate a 'struct' from the MIB compiler, so that there is no need to explicitly call a "get" function on each single configuration value, but one simply gets a C-language 'struct' with all the values filled in by the client library.</p> <p>As stated above, there should be some way how he MIB can notify applications about changes to certain nodes in the MIB tree, so the application[s] can react to that. I don't have a clear picture yet how transaction logic (like changing multiple values and then committing them at once) would work, or whether applications should even be able to reject/revert a modification?</p> <p>In either case, I just wanted to share my thoughts on this. I know it sounds rather complex, but I think the investment in some kind of unified configuration database system would save us of a lot of boilerplate code and subtle bugs and inconsistencies.</p> Cellular Network Infrastructure - Support #1719 (Stalled): review different 'getting started' gui... 2016-05-17T15:39:32Z laforge <p>There are plenty of different guides on how to compile + configure + run an OpenBSC based network. Some related to OsmoTRX based BTSs, some to nanoBTS, some ot E1 based BTSs.</p> Let's clean this up and divided it into <ul> <li>sections that are common among all configurations</li> <li>sections that are related to a specific BTS hardware (or hardware class)</li> </ul> <p>Also clearly indicate that building the software from source is only an optional step and refer to the OBS dpkg nightly builds as preferred method for regular users.</p> cellmgr-ng - Bug #1664 (New): m3ua connection should make non-blocking connect 2016-03-17T10:37:51Z zecke <p>The M3UA connection is made but the connect may block. This is nothing we want</p> <pre> connect(15, {sa_family=AF_INET, sin_port=htons(2905), sin_addr=inet_addr("A.B.C.D")}, 16 </pre> <p>There should already be examples for non-blocking connect.</p> Cellular Network Infrastructure - Bug #1612 (New): more configuration consistency checking 2016-02-23T16:22:14Z laforge <p>Right now it is possible to configure various things that wouldn't work, like an ARFCN that is outside the band of the BTS, or even the TRX.</p> <p>More consistency checks would make sense, together with reasonable error messages.</p> Cellular Network Infrastructure - Feature #1598 (New): Scripting language interface 2016-02-23T15:52:29Z laforge <p>For lab-type setups and testing of M2M devies, it would be useful to have some kind of script language bindings so the NITB could be instructed to perform certain actions under control of external software. Actions include triggering a hand-over, sending SMS, startin silent call, ...</p>